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CHAPTER  1 
Introduction 


1.  Introduction.  Program  offices,  operating  commands, 
and  test  organizations  have  been  directed  by  the  Chief  of 
Staff,  Air  Force  (CSAF),  to  employ  a  disciplined  test 
process  throughout  all  phases  of  an  armament/munitions 
life  cycle.  This  process  applies  to  all  testing  including 
developmental,  operational,  and  combined  testing.  The 
purpose  of  this  document  is  to  describe  a  disciplined 
process,  called  the  Armament/Munitions  Test  & 
Evaluation  (T&E)  Process,  and  provide  guidelines  for  its 
application  during  the  systems  acquisition  process  and 
throughout  the  life  of  the  system.  This  document 
supplements  Air  Force  Instruction  (AFI)  99-103,  Test  and 
Evaluation,  AIR  FORCE  TEST  PROCESS,  which  directs 
the  use  of  the  process. 

1.1.  Objectives.  The  objective  of  the 
Armament/Munitions  T&E  Process  is  to  standardize  the 
testing  and  evaluation  process  for  armament/munitions. 
Success  of  the  test  program  is  greatly  increased  with  a 
standardized  and  structured  T&E  process  based  on  a 
scientific  approach.  The  Armament/Munitions  T&E 
Process  is  applicable  through  all  phases  of  the  acquisition 
process;  concept  exploration/definition  through  operations 
and  support.  This  process  will  provide  a  T&E  audit  trail 
through  the  acquisition  process  in  addition  to  early 
identification  of  test  asset  requirements. 

1.2.  Application.  This  document  is  intended  for  program 
managers,  program  test  managers,  test  organization 


personnel,  MAJCOM  headquarters  staffs  and  others 
involved  in  the  management  of  T&E  for 
armament/munitions.  Contractors  should  use  this  guide  to 
become  familial*  with  the  Armament/Munitions  T&E 
Process.  Information  on  conducting  armament/  munitions 
T&E  is  available  from  the  Single-Face-to-Customer 
(SFTC)  Office,  Air  Force  Development  Test  Center 
(AFDTC/DRC),  Eglin  Air  Force  Base  (AFB),  Florida. 
Additional  information  on  conducting  Operational  T&E 
(OT&E)  is  available  from  the  Air  Force  OT&E  Center 
(AFOTEC/XR)  at  Kirtland  AFB,  New  Mexico.  All  Air 
Force  development  and  modification  programs  are 
expected  to  incorporate  the  principles  of  this  process  in 
their  test  planning  and  follow  its  guidelines  to  the 
maximum  extent  possible. 

2.  Background.  The  Armament/Munitions  T&E  Process 
and  the  directed  implementation  of  the  process  is  based 
upon  the  direction  given  by  the  CSAF  in  a  memorandum 
titled  “Air  Force  Test  and  Evaluation  (T&E)  Base 
Capability  -  ACTION  MEMORANDUM,”  dated  25 
January  1993.  The  memorandum  directed  the  establish¬ 
ment  of  a  center  of  expertise  for  armament/munitions  and 
a  disciplined  T&E  process  for  Air  Force  weapon  systems. 
AF/TE  then  directed  the  development  of  the  generic  Air 
Force  Test  Process  as  AFI  99-103  with  additional 
manuals  (see  figure  1)  for  Electronic  Warfare,  Armament/ 
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Munitions,  Command/  Control/Communications/Com¬ 
puters/  Intelligence,  Airframe-Propulsion-Avionics,  and 
Space.  SFTC  offices  were  established  to  develop  and 
support  the  early  application  of  the  test  process  in  five 
mission  areas  (table  1). 

2.1  Test  &  Evaluation  Process  Documentation 
2.1.1.  Air  Force  Policy  Directive  99-1,  Test  and 
Evaluation  Process.  This  policy  directive  outlines  a 
course  of  action  for  conducting  T&E  activities  during  the 
development,  production  and  deployment  of  Air  Force 
systems.  It  assigns  T&E  responsibilities  to  the 


implementing  command,  the  operating  and  supporting 
commands,  and  AFOTEC. 

2.1.2.  AFI  99-103  Test  Process.  This  instruction  directs 
implementation  of  the  Air  Force  Test  Process  in  general 
terms.  Several  Air  Force  documents  expand  the  direction 
and  application  of  the  Air  Force  Test  Process,  to  include 
those  specific  to  the  major  functional  areas  illustrated  in 
figure  1. 

2.1.3.  AFM  99-104,  Armament/Munitions  Test  & 
Evaluation  Process.  This  manual  provides  guidance  and 
procedures  for  implementing  the  Air  Force  Test  Process 
in  testing  Armament  and  Munitions. 


Figure  1.  Air  Force  Test  and  Evaluation  Process  Documentation. 
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Table  1.  Single-Face-to-Customer  Mission  Areas 

NAME 

ADDRESS 

AIRFRAME/PROPULSION/AVIONICS 

AFFTC/CAS 

EDWARDS  AFB  CA  93524-6842 

ARMAMENT/MUNITIONS 

AFDTC/DR 

EGLIN  AFB  FL  32542-5495 

COMMAND/CONTROL/COMMUNICATIONS/ 
COMPUTERS  AND  INTELLIGENCE 

AFDTC/DR 

EGLIN  AFB  FL  32542-5495 

ELECTRONIC  WARFARE 

AFDTC/DR 

EGLIN  AFB  FL  32542-5495 

SPACE 

SMC/CUC 

LOS  ANGELES  AFB,  CA  90009-2960 

3.  Scope.  This  manual  covers  the  armament/munitions 
mission  area.  Armament/munitions  described  in  this 
publication  are  defined  as  bombs  (guided  and  unguided), 


missiles  including  cruise  missiles,  guns,  ammunition,  and 
directed  energy  weapons.  Each  armament/munition 
subsystem,  such  as  the  seeker,  guidance  unit,  fuze, 
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warhead,  propulsion,  and  control  sections,  must  be 
individually  tested  to  verify  subsystem  performance  prior 
to  integration  into  and  testing  of  an  all-up-round. 
Armament/munitions  also  interact  with  the  launch 
platform.  Generally,  if  the  armament/munition  is  in 
development  or  being  modified  and  being  certified  for 
carry  on  a  mature  platform,  the  system  will  be  tested  via 
the  Armament/Munitions  Test  and  Evaluation  Process.  If 
however,  the  launch  platform  is  in  development  or 
undergoing  a  major  modification,  integration  of  inventory 
armament/munitions  is  tested  via  the  A-P-A  Test  and 
Evaluation  Process.  Integration  of  new  munitions  onto  a 
new  platform  will  be  tested  using  a  combination  of  the 
Armament/Munitions  and  A-P-A  Test  Processes.  This 
manual  excludes  nuclear  bombs  and  warheads. 


4.  Armament/Munition  Test  and  Evaluation  Commu¬ 
nity.  When  establishing  a  test  program,  all  stakeholders 
need  to  be  involved  in  the  early  test  planning  phase.  The 
stakeholders  include  the  user,  the  developer  (to  include 
the  program  office  test  manager),  the  contractor/sub¬ 
contractors,  the  sustainer,  the  trainers,  and  the  operational 
testers.  This  group  forms  the  nucleus  of  the  Test  Planning 
Working  Group  (TPWG)  which  will  develop  the  T&E 
Master  Plan  (TEMP).  Specifics  on  TEMP  content  are 
contained  in  AFI  99-101.  Other  test  organizations  and 
facilities  which  may  be  involved  with  the  TPWG  and  can 
offer  advice  on  specific  test  issues  are  discussed  in 
Section  C. 


CHAPTER  2 


Armament/Munitions  T&E  Process 


5.  Introduction.  Testing  is  conducted  during  the 
acquisition  cycle  to  acquire  data  which  are  analyzed  to 
evaluate  the  weapon’s  current  development  status  and 
risks  associated  with  the  continuation  of  the  development 
effort.  The  acquisition  cycle  is  shown  in  figure  2. 
Testing  supports  milestone  decisions  in  the  acquisition 
process  and  is  structured  to: 

Figure  2.  Testing  in  the  Acquisition  Cycle. 


(1)  Provide  essential  information  for  assessment 
of  acquisition  risk  and  for  decision  making; 

(2)  Verify  attainment  of  technical  performance 
specification  and  objectives; 

(3)  Verify  that  systems  are  operationally 
effective  and  suitable  for  intended  use. 
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6.  Air  Force  Test  Process.  Figure  3  depicts  the  Air 
Force  Test  Process.  The  process  applies  a  structured 
scientific  methodology  to  formulate  a  progressive  testing 


program  throughout  the  system  acquisition  life  cycle. 
The  test  process  begins  with  studying  the  test  objectives 
to  determine  the  expected  test  results,  so  that  measured 
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data  can  be  compared  and  analyzed  against  predicted 
values.  Once  testing  confirms  the  predicted  results, 
confidence  is  established  to  proceed  with  development  of 


that  system.  Testing  is  an  iterative  process  intended  to 
reduce  risk,  and  should  be  applied  to  each  step  of  system 
development. 


Figure  3.  The  Air  Force  Test  and  Evaluation  Process. 
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7.  Armament/Munitions  Test  Process  Description. 
The  relationship  of  the  Armament/Munitions  T&E 
process  to  T&E  resources  and  system  maturity  is  shown 
in  figure  4.  This  relationship  will  be  discussed  in  the 
following  sections.  The  first  face  of  the  cube  is  the  test 
process  (figure  5).  The  Armament/Munitions  T&E 
Process  utilizes  the  predict-test-compare  philosophy 
described  above,  and  tailors  it  to  the  Armament/Munitions 
application.  Digital  System  Models  (DSMs)  predict 
performance.  Differences  between  the  prediction  and  test 
results  are  explained,  and  the  DSM  is  updated.  The  Test 
Process  Archive  documents  decisions  relating  to  resource 
choices  and  differences  between  DSM  prediction  and 
actual  test  results.  The  following  is  a  brief  review  of  the 
test  process  described  in  AFI  99-103. 

•  Determine  test  objectives  -  What  is  to  be 
determined? 

•  Conduct  pretest  analysis  -  Predict  results. 

•  Test  -  Conduct  test  in  appropriate  T&E  resource 
facility  and  collect  data. 


•  Evaluate  test  data  -  Compare  with  prediction 

•  Report  results  -  Test  reports  and/or  briefings. 

7.1.  Determine  Test  Objectives.  The  first  step  of  the 
process  is  to  determine  the  test  objectives  for  the  program. 
Test  objectives  are  statements  of  system  performance 
with  respect  to  the  individual  characteristics  associated 
with  each  subsystem  function.  For  any  given  test  there 
may  be  a  large  set  of  possible  objectives.  These 
objectives  are  derived  from  operational  requirements 
defined  by  the  users,  which,  through  analysis,  are 
translated  into  technical  requirements  necessary  to  meet 
the  operational  need.  Sources  for  the  development  of  test 
objectives  are: 

•  Mission  Need  Statement 

•  Operational  Requirements 
Document/Requirements  Correlation  Matrix 

•  System  Threat  Assessment  Report 

•  Cost  and  Operational  Effectiveness  Analysis 

•  Concept  of  Operations 

•  Design  and  Performance  Specifications 
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Figure  5.  Armament/Munition  Test  and  Evaluation.Process. 
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7.1.1.  Test  Objectives  Terminology.  To  avoid 
confusion  and  to  convey  the  proper  meaning  of  test 
objectives,  the  following  terminology  should  be  used: 

•  Collect  -  Testing  to  collect  data  with  no  analysis 
or  evaluation. 

•  Compare  -  Testing  for  the  purpose  of  perceiving 
likeness  and  difference  in  test  items. 

•  Demonstrate  -  Testing  to  clearly  show  or  make 
evident  by  action  or  display.  Demonstration 
serves  as  conclusive  evidence  of  feasibility  or 
possibility  without  inference  to  expected 
behavior  or  performance. 

•  Determine  -  Testing  to  reveal,  recognize,  or 
establish  a  particular  characteristic,  trait,  or 
attribute. 

•  Evaluate  -  Testing  to  establish  worth 
(effectiveness,  suitability,  adequacy,  usefulness, 
capability)  of  a  test  item. 

•  Measure  -  Testing  to  make  a  quantitative 
determination. 

•  Verify  -  Testing  to  confirm  a  suspected  or  partly 
established  contention. 

7.1.2.  Test  Objective  Practicality.  Test  objective 
practicality  is  determined  by  at  least  considering  the 
following  topics: 

•  Achievability  -  Are  sufficient  measurement 
methods,  test  resources,  and  instrumentation 
available? 

•  Executability  -  Can  the  objectives  be 
accomplished  within  program  constraints  and 
limitations? 

•  Safety  -  Can  the  test  be  performed  safely? 

•  Utility  -  Do  the  test  objectives  clearly  and 
conclusively  evaluate  the  desired  feature? 

•  Cost  -  Can  the  customer  afford  the  cost  of  the 
objective? 

•  Schedule  -  Is  sufficient  time  available  to 
accomplish  the  objective? 

•  Environmental  Impacts  -  Can  the  objectives  be 
accomplished  without  adverse  effects  on  the 
environment? 

7.1.3.  Test  Objectives  Products.  The  products  of  this 
step  are  a  listing  of  Critical  Operational  Issues  (COIs), 
Measures  of  Effectiveness  (MOE),  and  Measures  of 
Performance  (MOP).  Also  formulated  during  this  process 
are  test  success  criteria  and  exit  criteria. 

7.1.4.  Test  Objectives  Lessons  Learned.  A  lesson 
learned  from  previous  programs  is  that  the  user 
sometimes  does  not  adequately  define  system 
requirements.  When  reviewing  requirements,  testers  must 
ensure  the  requirements  are  testable  and  clearly 
understood.  Ambiguous  requirements  not  identified  and 
corrected  early  in  the  acquisition  process  will  cause 
problems  later.  The  SFTC  offices  can  assist  in  correcting 
deficient  requirements  prior  to  the  Responsible  Test 


Organization’s  (RTO)  designation  through  early 
involvement. 

7.2.  Conduct  Pre-Test  Analysis  During  Develop¬ 
mental  T&E  (DT&E).  Analyses  are  performed  to 
determine  what  and  how  to  test,  as  well  as  to  predict  the 
outcome  of  System  Performance  Parameter  (SPP)  and 
Technical  Performance  Parameter  (TPP)  values.  The  type 
of  analysis  can  range  from  a  paper  analysis  where 
correlated  predicted  results  from  previous  tests  are 
available,  to  computer  simulations  using  a  validated 
digital  model.  Assumptions  related  to  the  test  will  be 
determined  and  documented.  The  analysis  will  help 
predict  the  outcome  of  the  test  and  identify  limitations  of 
the  test  procedures/capabilities.  The  product  of  pre-test 
analysis  is  a  detailed  test  plan  addressing  each  test 
objective.  The  Air  Force  Material  Command  (AFMC) 
Center  Directorate  of  Intelligence  will  work  with  the 
System  Program  Director  and  RTO  to  obtain  the  most 
current/detailed  threat  information  and  to  document 
requirements  for  intelligence  that  will  be  needed  during 
conduct  of  the  test.  The  detailed  test  plan  will  include  as 
a  minimum: 

•  Definition  of  each  test  objective. 

•  SPPs/TPPs  for  each  test  objective. 

•  Test  method/conditions  for  each  SPP/TPP. 

•  Detailed  instructions  for  each  test  participant. 

•  Predicted  outcome. 

•  Test  asset  requirements. 

•  Test  data  products  and  processing  requirements. 

•  Detailed  plan  for  utilization  of  test  results/data 
products. 

7.3  Conduct  Pre-Test  Analysis  for  Operational  Test 
and  Evaluation  (OT&E).  AFOTEC  has  developed  a 
new  test  concept  which  changes  the  approach  taken 
during  early  test  planning  to  develop  test  requirements 
and  scope  the  test  effort.  They  have  adopted  the  strategy- 
to-task  process  as  the  guiding  structure  for  T&E  strategy 
development.  AFOTEC  evaluations  should  consider  a 
system’s  ability  to  perform/support  its  operational  task(s) 
in  the  deployment,  employment,  and  sustainment  phases. 
The  rule  of  thumb  for  initial  test  concept  development  is 
to  structure  the  evaluation  at  the  highest  practical  level 
and  to  ensure  that  both  effectiveness  and  suitability  are 
addressed.  The  evaluation  should  be  focused  on  the 
system’s  contribution  to  accomplishment  of  its  associated 
task,  not  on  functions,  characteristics,  or  specifications. 
During  the  test  concept  development,  the  MOEs  and 
MOPs  are  developed.  MOEs  are  measures  of  either  how 
well  an  operational  task  is  accomplished  or  of  how  well 
an  operational  task  element  accomplishes  an  assigned 
tasking.  Documents  such  as  the  Operational  Require¬ 
ments  Document  (ORD),  Concept  of  Operations,  and 
Cost  and  Operational  Effectiveness  Analysis  provide  the 
necessary  reference  materials  to  develop  effectiveness 
issues.  MOPs  are  parameters  that  address  a  system’s 
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technical  characteristics  or  capability.  MOPs  can  be 
aggregated  to  evaluate  an  MOE  that  is  not  directly 
measurable.  For  more  information  concerning  OT&E  test 
planning,  contact  AFOTEC/XRC. 

7.4.  Conduct  Test.  Tests  are  conducted  utilizing  the 
most  appropriate  resource  categories.  The  six  resource 
categories  are  discussed  later  in  this  manual.  DT&E  is 
conducted  to  ensure  that  engineering  is  complete  and  that 
the  contractor  has  met  specifications.  Testing  is 
conducted  based  upon  a  plan  which  will  ensure  design 
problems  are  solved,  the  system  is  compatible  and 
interoperable  with  the  launch  platforms,  and  that  the 
system  is  ready  for  OT&E.  OT&E  is  conducted 
(primarily  by  AFOTEC)  to  determine  operational 
effectiveness  and  suitability  under  realistic  combat 
conditions.  Operational  users  operate  and  maintain  the 
system  in  conditions  as  close  as  possible  to  combat 
conditions.  During  the  test,  the  test  manager  will  monitor 
and  report  test  progress  to  senior  management,  resolve 
problems  that  arise  during  testing,  and  ensure  the  test  plan 
is  fully  executed. 

7.5.  Evaluate  Test  Data.  Test  data  are  processed  into  a 

form  that  will  allow  it  to  be  compared  to  pre-test 
predictions,  operational  requirements,  specifications,  and 
other  test  results.  The  results  are  analyzed  to  determine  if 
predicted  results  were  achieved  and  operational/technical 
requirements  have  been  satisfied. 


If  differences  between  measured  and  predicted 
performance  are  significant,  the  test  manager  must 
determine  if  one  or  more  of  the  following  occurred;  (1) 
the  test  design  was  flawed,  (2)  the  system  failed  to 
achieve  the  required  performance,  (3)  test  data  collected 
were  corrupted,  or  (4)  the  analysis/simulation  conducted 
during  pre-test  analysis  must  be  improved.  Computer 
models  and  digital  simulations  will  be  updated  after 
discrepancies  between  predicted  values  and  test  results 
are  resolved. 

7.6.  Report  Results.  Results  will  be  used  by  program 
management  in  support  of  internal  program  and  milestone 
decisions  by  higher  management.  Results  are  reported  in 
the  form  of  informal  and  formal  briefings,  memoranda, 
reports,  and  updates  to  the  TEMP  and  other  milestone 
decision  supporting  documentation.  System  deficiencies 
identified  during  this  test  step,  and  the  following 
evaluation  step,  will  be  documented  and  processed  in 
accordance  with  TO  00-35D-54  chapter  2.  The 
Deficiency  Reporting  (DR)  system  provides  a  systematic 
way  to  document  and  validate  problems.  Then  it  must  be 
used  to  investigate,  track,  and  resolve  problems. 

8.  Test  Resource  Categories.  The  second  face  of  the 
cube  (figure  4)  depicts  the  six  T&E  resource  categories. 
The  following  paragraphs  define  and  provide  examples  of 
the  kinds  of  tests  conducted  in  each  of  these  categories.  A 
greater  number  of  less  expensive  ground  tests  are 
accomplished  to  reduce  overall  test  cost  as  shown  in 
figure  6.  The  more  expensive  flight  tests  are  kept  to  a 
minimum.  A  more  detailed  discussion  on  facili¬ 
ties/capabilities  is  contained  in  Section  C. 


Figure  6.  Typical  Flow  of  Weapon  Testing. 


TIM  E 
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8.1.  Digital  Model  and  Computer  Simulation  Testing. 

This  category  provides  for  modeling  an  armament/muni¬ 
tions  system,  the  host  platform,  and/or  the  combat 
environment  and  executing  the  model  to  simulate  a  real 
world  event.  Computer  simulation  and  analysis  are  used 
prior  to  each  phase  of  testing  to  help  design  the  test  and, 
after  each  test,  to  extrapolate  test  results  to  other 
conditions.  In  contrast  to  past  practices  of  accomplishing 
only  design  trade-off  studies  during  concept  exploration, 
full  implementation  of  the  Air  Force 
Armament/Munitions  Test  Process  requires  development 
of  a  DSM  for  each  candidate  concept.  The  DSM  is  a  tool 
that  should  be  developed  to  help  in  system  design, 
performing  pre-test  analysis,  and  evaluating  the  results 
from  testing.  The  DSM  should  be  updated  as  the  system 
is  matured  and  maintained  throughout  the  life  cycle  of  the 
system.  All  digital  modeling  and  simulation  must  be 
verified,  validated,  and  accredited.  Verification  is  the 
process  of  determining  that  a  model  implementation 
accurately  represents  the  developer’s  conceptual 
description  and  specifications.  Validation  is  the  process 
of  determining  the  degree  to  which  a  model  is  an  accurate 
representation  of  the  real-world  from  the  perspective  of 
the  intended  uses  of  the  model.  Accreditation  is  the 
official  certification  that  a  model  or  simulation  is 
acceptable  for  use  for  a  specific  purpose.  For  more 
information  on  the  Verification,  Validation,  and 
Accreditation  (VV&A)  process  reference  DoD  Directive 
5000.59. 

8.1.1.  DSM  Background.  DSMs  vary  significantly  in 
their  level  of  detail  (and  thus  cost).  A  digital  system 
“description”  (i.e.,  model)  of  a  system  under  test  may  be 
as  simple  as  an  aircraft  flight  path  with  its  associated 
characteristics  (speed,  altitude,  acceleration)  and 
conditions  under  which  these  will  change,  what  they  will 
change  to,  and  their  rates  of  change.  Armament/ 
munitions  can  be  added  with  varying  degrees  of  detail 
depending  upon  system  test  requirements. 

DSMs  can  interact  with  other  models  at  various  levels  of 
detail  to  predict  a  system’s  estimated  performance.  For 
example,  a  DSM  for  an  air-to-air  missile  can  be  “flown” 
through  a  single  engagement  with  a  target  aircraft.  Many 
such  engagements  involving  numerous  threats  and 
perhaps  supporting  forces  can  be  combined  into  a  single 
model  to  simulate  a  mission  (i.e.,  a  mission  level  model), 
and  many  missions  can  be  simulated  in  a  higher  level 
model  to  simulate  combat  action  over  a  time  frame  (a 
campaign  level  model). 

The  DSM  will  normally  be  developed  by  the  program 
office  as  a  system  development  or  modification  contract 
deliverable.  It  models  the  proposed  system  design  or 
brassboard/production  hardware.  DSMs  are  developed  as 
part  of  the  system  engineering  process  to  support  system 
design,  analysis  and  testing  at  the  engineering,  platform, 
and  mission  levels,  as  appropriate.  If  new  components 
are  being  developed,  an  engineering-level  DSM  is  likely 


to  be  developed.  Platform  and  mission-level  DSMs 
should  be  developed  by  all  armament/munitions 
development  and  modification  programs.  The  DSM 
should  be  maintained  by  the  System  Program  Office  or 
system  manager  responsible  for  Air  Force  management  of 
the  system. 

If  there  is  no  DSM,  an  alternate  way  of  predicting  system 
pre-test  performance  must  be  used.  For  example,  this 
could  be  an  analog  model,  equations,  or  data  from  similar 
systems.  However,  the  DSM  is  a  powerful  tool.  The  lack 
of  one  will  put  an  extra  burden  of  responsibility  on  the 
program  manager.  Full  use  of  the  Armament/Munitions 
T&E  process  requires  having  a  DSM.  Not  having  a  DSM 
is  more  than  tailoring,  it  is  severely  constraining  the 
ability  of  the  T&E  process  to  work  as  intended. 

The  DSM  generally  consists  of  an  interlinked  system  of 
algorithms  which  model  the  response  of  the  arma¬ 
ment/munition  to  its  environment  from  launch  to  target 
intercept/impact.  The  DSM  addresses  the  complex 
interaction  of  the  various  facets  of  the  munitions  perfor¬ 
mance,  including  trajectory  analyses,  stability/con¬ 
trols/guidance  performance,  thermal-structures  response, 
and  propulsion  performance.  The  DSM  and  its  contribut¬ 
ing  components  must  be  verified,  validated,  documented, 
and  configuration  controlled  throughout  the  development 
process. 

The  program  office  will  keep  a  Test  Process  Archive.  It 
will  be  up-to-date  and  contain  adequate  data  to  establish 
predicted  and,  in  most  cases,  demonstrated  values  for 
SPPs  and  TPPs.  If  new  test  scenarios  are  developed  to 
respond  to  changes  in  defense  guidance  or  changes  to  the 
armament/munition  design  itself,  computer  simulations 
should  be  used  to  design  new  test  trials  and  predict  test 
results.  In  most  cases,  computer  simulations  are  adequate 
to  perform  a  pretest  analysis,  but  retesting  to  validate  key 
features  of  the  DSM  and  validate  subsequent 
armament/munition  performance  will  be  necessary. 

8.1.2.  DSM  Methodology.  During  concept  exploration, 
actual  test  data  for  the  complete  system  are  seldom 
available.  The  system  model  is  verified  and  validated  by 
comparing  it  to  a  model  of  a  similar  system  that  has 
demonstrated  fidelity.  The  DSM  provides  definition  of 
the  alternative  concepts  in  sufficient  detail  to: 

•  Conduct  the  conceptual  design  trade-off  studies 
of  the  competing  alternatives. 

•  Identify  key  SPPs  for  promising  candidates  and 
support  sensitivity  analyses  of  threat 
uncertainties. 

•  Support  sensitivity  analysis  of  tradeoffs  in 
performance  thresholds  and  objectives  of  key 
SPPs. 

•  Establish  initial  SPP  and  TPP  test  objectives. 

The  ability  of  candidate  system  concepts  to  provide  the 
desired  mission  capability,  as  stated  in  the  Mission  Need 
Statement  (MNS),  should  be  analyzed  using  DSMs  of  the 
system  concept.  Before  engineering  prototypes  are 
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developed,  a  more  detailed  version  of  the  DSM  which 
fully  models  the  complete  system  should  be  developed  by 
the  contractor  in  order  to  make  performance  predictions. 
However,  if  existing  hardware  components  that  have 
already  been  thoroughly  tested  are  going  to  be  used  for 
part  of  the  system,  these  components  can  continue  to  be 
modeled  for  analysis  of  system  design  and  performance. 
If  the  system  concept  is  very  complex  and  a  complete 
engineering  prototype  is  not  going  to  be  developed,  a  real¬ 
time  version  of  the  DSM  should  be  developed  to  support 
evaluation  of  the  prototype  components. 

8.1.3.  Types  of  Computer  Simulation.  Aside  from 
actual  launches/releases,  simulation  is  the  only  other 
means  to  represent  a  closed-loop  operation  of  the 
armament/munition.  The  following  types  of  computer 
simulations  are  available. 

8. 1.3.1.  Separation  and  Ballistics  Simulation.  These 
simulations  are  used  to  investigate  armament/munition 
separation  and  initial  trajectory  from  launch  until  the 
armament/munition  has  safely  cleared  the  aircraft.  They 
are  used  in  conjunction  with  launches  to  define  the 
aircraft/  armament/munition  launch  envelope. 
Armament/munition  separation,  from  first  movement  until 
clearance  of  the  aircraft,  is  a  critical  portion  of  flight.  The 
aircraft  flowfield,  in  conjunction  with  launcher  operation 
and  armament/munition  control,  determines  the  trajectory 
during  separation  and  thus  safe  separation  without 
degrading  the  armament/munition’s  performance.  The 
major  concerns  in  achieving  safe  separation  are: 

•  Armament/munition  striking  the  launch  aircraft 
during  non-powered  separation 

•  Armament/munition  striking  the  launch  aircraft 
while  powered 

•  Armament/munition  rocket  motor  plume 
impingement  on  the  launch  aircraft 

Determination  of  some  level  of  ballistics  accuracy 
(ballistic  coefficients  for  Operational  Flight  Programs 
{OFPs})  should  be  done  concurrently  with  development 
of  the  aircraft/  armament/munition  safe-separation  launch 
envelope.  AFI  63-104  defines  the  SEEK  EAGLE  (SE) 
process  which  can  provide  a  continuous  process  inclusive 
of  Wind  Tunnel  Testing  (WTT)  necessary  to  accomplish 
safe  separation  and  ballistic  accuracy  analysis.  This 
process  requires  a  reasonably  high  fidelity  aerodynamic 
definition  of  the  armament/munition  under  development 
and  provides  the  potential  for  reduced  cost  and  time  for 
armament/munition  certification  on  an  operational 
aircraft.  The  concept  of  developing  ballistics  algorithm 
coefficients  from  ground  based  simulation  models  serves 
two  purposes:  1)  provides  a  preliminary  indication  of 
Circular  Error  Probability  for  the  armament/muni¬ 
tion/aircraft  being  integrated,  and  2)  provides  the  potential 
for  reduced  certification  cost  and  time  via  the  replacement 
of  ballistics  flights  typically  required  for  ballistics 
coefficient  development. 


8. 1.3.2.  Trajectory  Simulation.  This  simulation 
computes  armament/munition  forces  and  moments  to 
define  trajectory  from  launch  to  target  impact/intercept. 
The  simulation  incorporates  measured  data  (such  as  wind 
tunnel  data),  armament/munition  data,  atmospheric  data 
and  kinematic  equations  of  motion  to  predict  the 
armament/munition  trajectory.  The  6-degree-of- freedom 
(DOF)  digital  trajectory  simulation  provides  an  excellent 
representation  of  armament/munition  performance 
unaffected  by  detailed  guidance  and  target  considerations. 
When  the  requirement  for  many  flight  simulations  over  a 
wide  range  of  launch  conditions  and  other  flight 
parameters  exists,  a  “fast”  trajectory  simulation  is  used.  It 
is  faster  and  cheaper,  however,  some  accuracy  is 
sacrificed.  A  “fast”  trajectory  simulation  is  achieved  by 
simplifying  detailed  trajectory  simulation  models  to  obtain 
the  desired  run  time,  while  maintaining  an  acceptable  level 
of  simulation  accuracy.  The  “fast”  trajectory  simulations 
use  fewer  degrees  of  freedom  and  more  simplified 
assumptions. 

8. 1.3.3.  Lethality  Simulation.  This  is  an  analytical 
representation  of  the  armament/munition  and  its 
interaction  with  the  target.  This  simulation  allows  muni- 
ion  effectiveness  calculations  to  be  performed  and 
includes  fuzing  performance,  warhead  blast/fragmentation 
effects,  target  vulnerability/survivability  and  warhead/ 
target  interactions.  To  perform  this  task  requires 
meticulous  accumulation  of  target  damage  assessment 
data  from  combat  experience,  comprehensive  testing,  and 
intelligence  sources.  Once  this  data  has  been  collected,  it 
can  be  transformed  into  applicable  armament/munition 
effectiveness  data  with  the  aid  of  mathematical  modeling 
and  computers.  The  data  may  be  used  to  assess  the 
viability  of  an  armament/munition  system,  determine  the 
payoff  of  a  new  tactic,  or  provide  a  realistic  estimate  of 
the  armament/  munition  expenditures  required  to  achieve 
a  desired  damage  level  on  a  selected  target. 

8. 1.3.4.  Armament/Munition  Response  Simulation. 
An  important  role  for  the  DSM  is  to  model  the  response 
of  the  armament/munition  itself  to  the  environment  it 
experiences  from  launch  to  target.  The  response  of  the 
armament/munition  to  the  external  environment  must  be 
considered,  including  the  effects  of  aerodynamically 
induced  pressure  and  heat,  rain,  dust,  radiation,  etc.  The 
armament/munition  model  must  also  include  the  effects  of 
structural  vibration,  control  systems  performance,  seeker 
system  performance,  propulsion  system  performance,  etc. 
The  model  is  generally  an  integration  of  subelement 
algorithms,  including  structural  models  of  components 
and  subsystems,  models  of  environmental  parameters 
including  aerodynamics  and  weather/dust,  and  an 
integration  of  the  subelements  to  produce  an  overall 
system  model.  Models  are  used  extensively  in  designing 
the  armament/munition’s  subsystems,  structure,  etc.,  to 
achieve  necessary  performance  while  ensuring 
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armament/munition  integrity  throughout  its  envelope  of  are  available,  including  glint,  Radar  Cross  Section  (RCS) 

operation.  scintillation,  and  aspect  dependent  RCS.  In  addition,  a 

8.1.4.  Simulation  Tools.  broad  range  of  ECM  types  are  available  and  can  be 

8.1.4. 1.  Emulation.  In  the  early  phases  of  an  acquisition  combined  on  a  single  target  or  distributed  across  a  range 

program,  the  contractor  must  initiate  the  design  of  the  of  targets.  Ground  clutter  is  presented  at  the  proper  range 

armament/munition  computer  program  before  the  and  Doppler,  and  has  the  correct  shape,  bandwidth,  and 

processor  is  available.  The  computer  program  is  angle.  The  Complex  Seeker  Simulation  has  undergone 

developed  on  a  computer  referred  to  as  an  emulator.  The  extensive  validation,  both  at  the  functional  (model)  level 

emulator  is  designed  to  function  exactly  the  same  as  the  as  well  as  at  the  performance  (probability  of  guidance) 

armament/munition  digital  processor  and  is  used  as  a  level. 

design  and  diagnostic  tool  for  debugging  software.  The  8.1.5.  Typical  Digital  Model  and  Computer 

emulator  can  maintain  and  monitor  a  baseline  Simulation  Testing.  Table  2  shows  the  typical  modeling 

armament/munition  software  configuration.  and  simulation  tests  a  new  armament/munition  would 

8. 1.4.2.  Hardware-In-the-Loop  (HITL)  Simulation.  undergo. 

There  is  a  limit  to  what  can  be  accomplished  with  8.2.  Measurement  Testing.  Examples  of  measurement 

analytical  simulation.  HITL  simulation  has  evolved  data  collected  are  aerodynamic  loads/coefficients,  target 

because  of  the  need  to  predict  armament/munition  signature,  background,  clutter  characterization,  and  RCS. 

performance  as  a  function  of  sophisticated  guidance,  Early  brassboard  or  prototype  hardware  characterization 

tracking  and  signal  processing  activities.  facilitates  verification  of  system  models.  Prototype 

Armament/munition  hardware  is  integrated  into  the  hardware  may  be  shipped  to  measurement  facilities  in 

simulation  to  obtain  valid  analytical  representation  of  the  order  to  measure  system  parameters  such  as  aerodynamic 

seeker,  guidance,  and  control  subsystem.  HITL  drag,  accuracy,  RCS  or  IR  signature.  These  types  of  data 

simulations  exercise  actual  hardware  (seeker,  sensor,  are  needed  as  inputs  to  computer  simulations.  The  use  of 

guidance,  autopilot,  etc.)  through  stimulation  of  measurement  facilities  later  in  the  program  is  much  more 

armament/munition  system  sensors  and  simulated  forces  useful.  With  systems  installed  on  expected  platforms, 

of  motion.  HITL  analysis  allows  seeker,  guidance,  and  valid  measurements  can  begin  to  be  taken  which  can 

control  software  and  hardware  to  be  evaluated  and  confirm  design  capabilities,  identify  design  problems,  and 

validated  prior  to  actual  free  flight.  HITL  can  include  the  determine  employment  options.  Measurement  facility 

total  armament/munition  system  or  critical  subsystems.  testing  should  eventually  establish  values  for  TPPs  for  all 

8. 1.4.3.  Computational  Fluid  Dynamics  (CFD).  CFD  is  mission  critical  system  variables.  Specific  examples  of 

a  technology  used  as  a  production  tool  to  supplement  such  testing  are  measurement  of  inertial  guidance  system 

WTT  and  enhance  the  aircraft  stores  certification  process.  error  and  aging  and  surveillance  testing  of  a  rocket  motor. 

Specific  applications  are  for  safe  separation  and  ballistics 

accuracy  as  well  as  definition  of  loads  on  stores.  Testing  inquiries  should  include  developing  a  plan  for 

8. 1.4.4.  Complex  Seeker  Simulation.  The  Complex  direction  and  guidance  in  establishing  the  computational 

Seeker  Simulation  is  used  to  quantify  Electronic  and  testing  requirements.  The  concept  of  developing  a 

Countermeasures  (ECM)  performance  and  predict  flight  reasonably  high  fidelity  aerodynamic  simulation  model 

test  results,  as  well  as  evaluate  hardware  and  software  must  be  stressed  to  establish  structural  loads,  control 

upgrades  to  air-to-air  missiles.  This  digital  simulation  actuation  and  rate  requirements,  and  mission  performance 

accurately  represents  the  missile’s  tactical  software  acceptability.  This  can  be  accomplished  where 

execution  by  emulating  the  hardware  data  processor  and  computational  driven  aerodynamic  models  complement 

reading  the  actual  missile  program  memory.  Missile  and  add  to  the  data  obtained  from  measurement  facilities, 

receiver  hardware  Inertial  Reference  Unit,  antenna  Table  3  shows  the  measurement  tests  an  armament/ 

tracking  and  stabilization,  and  missile  equations  of  motion  munition  would  undergo. 

are  all  modeled  in  Fortran  and  interact  with  the  tactical 

software  execution.  A  variety  of  target  radar  signatures _ 

Table  2.  Digital  Models  and  Computer  Simulation  Testing. 

ALL-UP  ROUND  3,  4,  5,  6  -  DOF  Performance  Stability  and  Control 

-  Range  Fit  Check 

-  Accuracy  Target  Signature 

-  Trajectory  Cross  Section 

-  Separation  Endgame  Lethality 

Logistics  Support  Electromagnetic  (EM)  Compatibility 

Aging/Surveillance _ Weapon  System  Effectiveness _ 
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Static/Dynamic  Loads 

SEEKER/SENSORS 

Scene  Simulation 

Target/Threat  Mode 

High  Fidelity  Simulation 
Counter-countermeasures 

Countermeasures 

GUIDANCE  &  CONTROL 

Stability  and  Control 

AIRFRAME/PROPULSIO 

Cross  Section  Models 

3-D  Hydrodynamic  Codes 

N 

Infrared  (IR)  Signature 

Computational  Fluid  Dynamics 

FUZE 

Proximity  Sensing 

EM  Compatibility 

Interoperability 

WARHEAD 

Lethality 

Shape  Charge  Models 

Warhead  Performance 

Fragmentation  Dispersing  Pattern 

Velocity  and  Impact  Pattern 

3-D  Hydrodynamic  Codes 

Kinetic  Energy  Penetration 

8.3.  Integration  Laboratory  Testing  .  This  testing  is 
performed  to  evaluate  individual  armament/munition 
components  or  multiple  components  and  their  interaction 
with  each  other  in  a  controlled  environment.  A  variety  of 
computer  simulations  and  test  equipment  are  used  to 
generate  scenarios  and  environments  to  test  the 
component(s)  for  reliability,  safety,  and/or  functional 
performance.  In  some  cases,  additional  real-time  digital 
simulations  may  be  piggybacked  on  laboratory 
technology  development  programs  to  verify  fairly 
complex  concepts  that  require  sensor  fusion  or  involve 
aircrew  displays  and  operator  actions  that  need  to  be 
evaluated  with  real-time,  man-in-the-loop  simulations. 
Test  data  from  on-going  technology  demonstrations 
should  be  correlated  with  the  computer  simulation 
analyses. 

If  additional  testing  is  conducted,  computer  simulation 
should  be  used  as  appropriate,  to  conduct  pretest  analysis 
to  establish  desired  test  conditions  and  predict  expected 
results.  Testing  begins  in  the  contractor’s  System 
Integration  Laboratory  (SIL)  and  other  test  facilities 
before  any  components  are  shipped  for  testing  in 
Government  facilities.  This  testing  should  focus  on 
identifying  hardware  and  software  problems,  maturing 
system  performance,  and  evaluating  projected  reliability 
and  maintainability  levels. 


The  DSM  and  environment  simulation  should  be  used  to 
accomplish  the  pretest  analysis  and  help  structure  the  test 
trials.  A  number  of  tests  can  be  conducted  at  the 
contractor’s  SIL  to: 

•  Identify  and  correct  software  errors. 

•  Determine  required  bus  capacities. 

•  Evaluate  system  timing  and  input/output 
performance. 

•  Evaluate  aircrew  controls  and  displays  format. 

•  Evaluate  built-in-test  fault  isolation. 

•  Observe  system  performance  degradation. 

The  engineering  development  will  stimulate  components 
of  the  engineering  prototype  to  evaluate  performance  and 
compliance  with  technical  requirements.  This  testing 
should  repeat  the  tests  conducted  during  the  previous 
phase  to  confirm  that  performance  thresholds  have  been 
achieved  and  to  correct  any  identified  hardware  and 
software  problems.  Once  the  components  of  the  system 
under  test  have  been  tested  in  the  contractor’s  facilities, 
they  can  be  shipped  to  Government  test  facilities  for 
further  testing.  The  amount  of  SIL  testing  needed  in  the 
production  phase  will  depend  on  the  number,  extent,  and 
complexity  of  changes  to  the  system.  Table  4  shows  the 
integration  laboratory  tests  a  new  armament/munition 
would  undergo. 


Table  3.  Measurements  Testing. 

ALL-UP  ROUND 

Structural 

Static/Dynamic  Loads 

Aerothermal 

Non-Destructive  Inspection/Testing 

Signatures 

EM  Compatibility 

Environmental 

Safety 

Weight,  Balance,  Physical  Characteri sties 

SEEKER/SENSORS 

Mechanical/Electrical  Interfaces 

Countermeasure  Effects 

Susceptibility  to  Threats 

Target  Recognition,  Acquisition  and  Track 

Clutter/Background  Rejection 

GUIDANCE  & 

Accuracy 

Initial  Reference 

CONTROL 

Target  Recognition,  Acquisition  and  Track 

Clutter/Background  Rejection 

AIRFRAME/ 

Cross  Section  Models 

Rocket  Motor  Burn/Thrust 
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PROPULSION 

Structural  Airbreathing  Engine  Thrust, 

Aerothermal  Energy  Management 

IR  Signature 

FUZE 

Proximity  Sensing  Timer  Performance 

Initiator  Performance  EM/Acoustic  Compatibility 

Target  Discrimination  Impact 

Environmental  Penetration 

-  Shock  Survivability 

-  Vibration 

-  Drop 

-  Sand/Dust/Humidity/Temperature 

WARHEAD 

Arena  Fuze  Integration 

Fragment  Dispersion  Pattern,  Velocity,  Sympathetic  Detonation 

Impact  Pattern  Non-Destructive  Inspection/Testing 

Table  4.  Integration  Laboratory  Testing. 

ALL-UP  ROUND 

Functional  Performance 

Safety 

Embedded  Software 

SEEKER/SENSORS 

Target  Recognition,  Acquisition  and  Track 

Functional  Performance 

GUIDANCE  &  CONTROL 

Software  Integration 

Software/Hardware  Interface 

Functional  Performance 

AIRFRAME/PROPULSION 

Motor  Ignition  and  Bum 

FUZE 

Electronic  Function 

Mechanical  Function 

Safety 

WARHEAD 

Warhead/Fuze  Integration 

Functional  Performance 

8.4.  Hardware-In-the-Loop  Testing.  This  is  ground 
testing  that  involves  the  armament/munition  system 
hardware  in  a  closed-loop  mode  against  high-fidelity 
target  and  threat  simulations.  This  testing  allows 
developmental  and  production  systems  to  be  tested  under 
controlled  and  repeatable  test  conditions,  thus  providing  a 
less  expensive  complement  to  flight  testing.  During  the 
Concept  Exploration  and  Development  (CE&D)  Phase,  it 
is  preferable  to  evaluate  the  system  concept  in  HITL 
facilities.  However,  only  a  few  system  concepts  will  be 
advanced  enough  in  this  phase  to  have  evolved  prototype 
hardware.  If  prototype  hardware  is  available,  HITL 
facilities  should  also  be  used  to  refine  designs,  check 
system  performance  in  operational  environments,  and 
work  out  problems  encountered  before  formal,  contractual 
testing  monitored  by  the  government. 

When  contractor  SIL  testing  has  been  completed, 
engineering  prototype  components  should  be  transferred 
to  HITL  facilities.  Initial  testing  in  Government  facilities 
should  be  contractor  conducted  so  the  contractor  is 
afforded  the  opportunity  to  work  against  a  rigorous  test 
environment  and  mature  the  system  in  a  non-adversarial 


manner.  The  main  thrust  of  HITL  testing  is  to  evaluate 
the  performance  of  the  system  under  test  by  simulators. 
Types  of  HITL  activities  conducted  during  demonstra¬ 
tion/validation  include: 

•  System  trade  studies. 

•  Preflight  simulation  to  verify  release  conditions 
and  mission  profiles. 

•  Predict  in-flight  performance. 

•  Algorithm  development,  maturation. 

•  Countermeasure  susceptibility. 

•  Flight  test  planning. 

•  Operational  MOEs  and  MOPs. 
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The  production  prototype  components  are  usually  tested 
in  Government  HITL  test  facilities  before  the  system  is 
installed  in  a  testbed  or  dedicated  test  aircraft  for  OAR 
testing.  This  ground  testing  should  focus  on  confirming 
that  problems  identified  in  earlier  testing  have  been  fixed 
and  performance  thresholds  can  be  achieved.  Because  a 
complete  system  is  available  for  this  testing  compared  to 
just  critical  components  available  during  demonstra¬ 
tion/validation  testing,  this  is  the  first  opportunity  to 
conduct  integrated  system  effectiveness  tests.  Specific 
tests  to  be  conducted  will  depend  on  the  functions 
included  in  the  system.  Some  examples  are: 

•  Pre-Flight  simulations  to  verify  release 
conditions  and  mission  profiles. 

•  Flight  simulations  to  assess  in-flight 
performance  in  adverse  weather. 

•  Flight  simulations  to  assess  in-flight 
performance  in  the  presence  of  multiple  targets 
and  countermeasures. 


8.5.  Installed  Systems  Testing  (1ST).  1ST  involves  the 
entire  armament/munition  and  typically  its  integration 
with  a  host  platform.  Electronic  linking  of  test  facilities 
may  be  used  to  provide  expanded,  more  realistic  test 
conditions  than  those  available  using  a  single  resource. 
The  armament/munition  may  or  may  not  be  physically 
installed  on  the  host  platform.  1ST  normally  provides  the 
first  opportunity  to  evaluate  system  operation  on  a 
armament/munition  system  platform.  This  testing  is 
conducted  to  evaluate  the  integrated  performance  of  the 
armament/munition  as  part  of  a  armament/munition 
system  platform.  The  purpose  of  1ST  is  to  verify 
functional  (electronic,  logical  and  mechanical) 
compatibility.  By  linking  HITL  and  1ST  facilities,  total 
armament/munition  performance,  from  initialization  to 
fuzing,  can  be  verified  in  a  non-destructive  environment 
prior  to  flight  test.  Some  examples  are: 


•  Post  flight  simulation  to  reproduce  field  events 
for  analysis  and  troubleshooting.  If  an 
armament/munition  system  problem  is  found  in  a 
HITL  facility  and  fixed,  it  can  be  verified  before 
flight  via  HITL  simulation  of  the  event  that 
caused  it. 

HITL  simulations  are  used  to  establish  arma¬ 
ment/munition  systems  performance  during  CE&D, 
Demonstration/Validation  (Dem/Val),  and  Engineering 
and  Manufacturing  Development  (E&MD).  HITL  test 
facilities  represent  a  considerable  investment  which  can 
be  applied  directly  to  nondestructive  surveillance  and 
shelf  life  testing  during  the  production  and  deployment 
phase.  Table  5  shows  the  HITL  tests  an 
armament/munition  would  undergo. 


•  1ST  Facility  (ISTF)  -  Verification  of  target 

reference  and  other  targeting  data  from  aircraft  to 
armament/munition.  GPS  initialization  and 
transfer  alignment  data  from  aircraft  to 


Table  5.  Hardware-in-the-Loop  Testing. 

ALL-UP  ROUND 

6-DOF  Verification  System  Performance  Verification 

Embedded  Software  -  Guidance 

Interoperability  With  Aircraft  -  Lethality 

-  Weapon  System  Effectiveness 

SEEKER/SENSORS 

Seeker  Integration  Target  Recognition,  Acquisition,  and  Tracking 

Countermeasures  Interoperability  With  Aircraft 

Counter-Countermeasures  Performance  Verification 

Flight  Test  Focusing  Global  Positioning  System  (GPS) 

Inertial  Measurements  Function/Performance 

Prototype  Comparison  6-DOF  Verification 

Targeting  Endgame  Clutter/Background  Rejection 

GUIDANCE  &  CONTROL 

Performance  Verification  Target  Recognition,  Acquisition,  and  Tracking 

6-DOF  Verification  Flight  Test  Focusing 

Guidance  &  Control  Unit/Inertial  Aero  Loads 

Measurement  Unit/GPS  Clutter/Background  Rejection 

Interoperability 

Interoperability  With  Aircraft 

AIRFRAME/PROPULSION 

N/A 

FUZE 

N/A 

WARHEAD 

N/A 
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armament/munition.  Transfer  of  fire  control 
information  to  armament/munition  prior  to 
launch. 

•  Linked  ISTF  and  HITL  Testing  - 

Armament/munition  initialization  and  launch 
sequence  are  accomplished  using  an  aircraft  in 
the  ISTF.  Information  is  transferred  to  the 
armament/munition  over  the  link.  A  munitions 
flight  simulation  is  completed  in  the  HITL 
facility. 


1ST  continues  during  the  Production  and  Deployment 
Phase  to  evaluate  the  installed  system’s  interfaces  and 
interoperability  with  other  aircraft  systems.  The  actual 
configuration  of  the  armament/munition  to  be  deployed  is 
tested  using  installed  system  ground  test  facilities.  These 
facilities  are  also  used  to  perform  pretest  checkout  before 
testing  on  Open  Air  Ranges  (OARs).  This  procedure 
helps  in  identifying  problems,  saving  flight  test  hours,  and 
producing  more  usable  test  data.  Table  6  shows  the  1ST  a 
new  armament/munition  would  undergo. 


Table  6.  Installed  Systems  Testing. 

ALL-UP  ROUND 

Aircraft  Integration 
EMI/EMC/HERO/Rad  Haz 
Transfer  Aignment 

Target/Retargeting 

Data  Link  Performance 

Pylon/Rack  Ejection 

SEEKER/SENSORS 

Seeker  Integration 

Electronic  Counter- 
Countermeasures  (ECCM) 
Testing 

Data  Link  Transfer 

EMI/EMC 

Endgame  Targeting  Hand-off 

Antenna  Measurement 

Target  Detection/Recognition/Tracking 

GUIDANCE  &  CONTROL 

G&C  Integration 

Miss  Distance 

Countermeasure  Testing 

T  arget/Re-T  ar  geting 

Captive  Carry  Testing 

Software  Agorithms 

Hardware/Software  Integration 

Target  Recognition,  Acquisition,  and  Tracking 

AIRFRAME/PROPULSIO 

N 

Mechanical/Electrical  Fit  Checks 

FUZE 

EMI/EMC  Testing 

Proximity  Function 

Aming  function 

WARHEAD 

N/A 

8.6.  Open  Air  Range  Testing.  This  testing  involves 
open  air  test  ranges  for  the  purpose  of  gathering  data  for 
evaluating  the  armament/munition  under  natural 
environment  operating  conditions,  for  validation  of  and 
updating  models,  and  simulations.  Testing  may  be 
ground  or  airborne  with  emphasis  on  providing  an 
environment  of  real  world  phenomena  up  to  and  including 
live  armament/munitions  against  real  targets  and  threats. 
If  advanced  technology  development  programs  have 
already  produced  brassboard  hardware,  the  laboratories 
may  be  conducting  flight  tests  by  installing  the  hardware 
in  airborne  testbeds.  As  was  the  case  with  the  ground- 
based  technology  demonstration  testbeds,  there  may  be 
opportunities  to  obtain  test  data  from  laboratory  flight 
testing  to  check  equipment  operation  and  help  verify  the 
computer  simulations  used  in  CE&D.  Airborne  testing  is 
required  early  in  a  system’s  development  or  modification 
cycle  in  order  to  evaluate  its  achieved  performance. 
During  the  Dem/Val  phase,  this  can  be  accomplished  by 
installing  an  engineering  prototype  of  components  or  the 
proposed  armament/munition  in  a  bench  configuration, 
aboard  a  large-body  testbed  aircraft  or  installing 
components  in  a  pod/missile  shape  on  a  fighter  aircraft  (or 


on  a  highly  instrumented  rocket  sled,  when  precision 
position  and  velocity  data  are  of  interest).  OAR  tests  are 
used  to: 

•  Evaluate  Human  Factors  in  interfacing  with 
prototype  hardware  controls  and  displays. 

•  Verify  that  prototype  hardware  works  under 
actual  flight  conditions. 

•  Experiment  with  alternative  employment  tactics. 
Flight  testing  during  E&MD  constitutes  the  first 
opportunity  to  measure  selected  SPPs  and  TPPs  of  the 
armament/munition  in  the  actual  operating  environment 
of  its  host  armament/munition  system  platform.  It 
provides  the  means  to  calibrate  the  other  classes  of 
facilities  (i.e.,  digital  simulations,  SILs,  HITL  facilities, 
1ST  facilities)  and  to  validate  the  predictions/ 
measurements  made  therein.  By  so  doing,  it  establishes 
an  acceptable  confidence  factor  for  all  SPPs  and  TPPs. 
Some  examples  of  E&MD  OAR  testing  are: 

•  Conduct  SEEK  EAGLE  certification  tests. 
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•  Verify  technical  and  system  performance 
requirements. 

•  Conduct  live  fire  tests. 

•  Verify  platform  parameters  in-flight. 

•  Collect  data  on  component  failure  rates  for  R&M 
analysis. 

•  Evaluate  software  capabilities. 

For  Initial  Operational  T&E  (IOT&E),  OAR  testing  is 
indispensable  for  evaluating  operational  effectiveness  and 
suitability  because  it  provides  the  final  basis  for 
comparison  of  previously  collected  data  and  a  point  of 


Table  7.  Open  Air  Testing. 

ALL-UP  ROUND 

Captive  Carry 

Flutter/Loads 

One-on-One  Flight  Test 

Jettison  Test 

EMI/EMC  Compatibility 

Weapon  System  Effectiveness 

Separation  Test 

Targeting/Retargeting 

Live  Launch 

Data  Link  Updates 

Many-on-Many  Flight  Test 

ECM/ECCM 

SEEKER/SENSORS 

Comparative  Seeker  Evaluation 

Clutter/Background  Rejection 

GUIDANCE  &  CONTROL 

Target  Recognition,  Acquisition,  and  Tracking 

Gain  Scheduling 

Clutter/Background  Rejection 

Autolock 

Accuracy 

Manual  Track 

AIRFRAME/PROPULSIO 

N 

Flight  Stability 

Motor  Ignition  and  Bum 

FUZE 

Proximity  Sensing 

Impact  Functioning 

Delay  Function 

Penetration  Survivability  and 

Warhead  Initiation 

Functioning 

WARHEAD 

Lethality 

Penetration 

Blast/Fragmentation 

Performance  and  Effects 

departure  for  additional  simulation,  analysis  and 
evaluation.  OAR  testing  is  required  to  determine  if  the 
production  configurations  of  the  armament/munition 
satisfies  user  requirements.  These  tests  should  confirm 
whether  armament/munition  performance  requirements 
established  in  previous  phases  have  been  achieved. 
During  the  sustainment  phase,  open  air  aging  and 
surveillance  testing  is  accomplished.  Table  7  shows  the 
open  air  tests  a  new  air-to- surface  guided 
armament/muni-tion  would  undergo. 


8.7.  T&E  Resource  Categories  in  the  Acquisition 
Process.  Figure  7  shows  how  the  six  T&E  Resource 
Categories  are  used  during  the  life  cycle  of  an  arma¬ 
ment/munition. 

9.  System  Maturity.  The  third  face  of  the  cube  (figure 
4)  illustrates  the  system  maturity  concept  which  must  be 
applied  to  the  test  program.  Initial  testing  is  carried  out 
on  components.  Components  are  then  assembled  into  the 
various  subsystems,  which  are  tested  prior  to  integration 
into  the  armament/munition  system.  This  section  will 

Figure  7.  Armament/Munition  Life  Cycle  (Notional). 


discuss  the  various  levels  of  testing  and  associate  facilities 
and  techniques. 

9.1.  Concept  Phase  Testing.  During  the  conceptual 
phase  of  an  armament/munition  program,  the  mission 
shortfall  described  in  the  MNS  is  evaluated  and  various 
solutions  to  satisfy  the  deficiency  are  studied.  During  the 
conceptual  phase,  trade  studies  would  be  accomplished 
using  digital  models  and  computer  simulation  testing  to 
determine  the  optimum  design  solution.  Once  the 
configuration  of  the  new  system  is  defined,  preliminary 
measurement  testing  may  be  accomplished  if 
representative  hardware  is  available. 
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9.2.  Component  Testing.  The  basic  thrust  of  component 
testing  is  to  ensure  the  components  perform  within 
subsystems  as  designed.  Once  the  initial 
armament/munition  component  and  subsystem 
requirement/design  has  been  determined,  component 
testing  begins.  The  test  process  is  applied  to  prototype 
components  and  actual  components  using  various  levels 
of  test  resources.  The  test  resources  chosen  for  a  given 
component  depend  on  factors  such  as  cost,  design  risk, 
integration  requirements,  maturity  of  system  in  which  it  is 
embedded,  simulation  fidelity,  and  schedule.  The 
facilities  used  to  conduct  component  testing  generally 
include  digital  modeling  and  simulation  facilities, 
measurement  facilities,  SILs,  and  some  testing  in  HITL 
facilities.  Generally  the  first  three  categories  of  facilities 
are  owned  and  operated  by  contractors/  subcontractors 
and  the  testing  is  done  by  them  with  Government 
oversight  and  review. 

Models  used  during  component  development  are  the  same 
models  initially  used  to  refine  the  basic  design. 
Components  are  generally  modeled  within  subsystem 
models  and  represent  component  performance  to  the 
required  fidelity.  Stand-alone  component  models  are 
created  if  the  component  requirements  push  the  state-of- 
the-art  capabilities  or  if  they  must  very  accurately 
represent  the  component. 

As  part  of  post-test  analysis,  test  results  are  compared  to 
predicted  results  from  models  to  ensure  components 
function  as  designed  and  meet  all  requirements. 

The  result  of  component  testing  is  that  the  given 
component  is  qualified  for  the  next  series  of  tests,  which 
is  subsystem  development  testing. 


9.3.  Subsystem  Testing.  During  the  subsystem 
development  phase,  the  number  of  new  models  created 
begins  to  taper  off,  but  model  sophistication/complexity 
continues  to  increase  as  a  system  develops.  Similarly,  test 
sophistication/complexity  has  increased  from  that  taking 
place  during  component  development.  Subsystem  testing 
is  primarily  conducted  in  SILs  and  HTTLs.  These 
facilities  assess  how  well  a  given  subsystem  functions 
against  design  requirements  and  also  how  well  it 
integrates  with  other  subsystems.  Detailed  failure  modes 
and  effects  analysis  is  typically  performed  at  this  time  and 
during  system  testing  using  these  facilities.  This  is 
accomplished  by  inducing  a  set  of  test  matrix  failures  and 
observing  the  effect  on  the  subsystem  or  system.  The 
types  of  induced  failures  range  from  single  input/output 
signals  within  a  component  or  single  component  failure  to 
multiple  component  failures. 

During  subsystem  development,  integration/interfacing 
becomes  testable  to  a  limited  extent.  Similar  to 
extrapolating  component  test  results  to  assess  how  well 
they  meet  user  requirements,  subsystem  testing  is  the  first 
opportunity  to  directly  assess  system  integration  without 
excessive  extrapolation.  Usually  subsystem  testing  is 
planned  and  conducted  by  the  contractor  with 
Government  involvement.  Facilities  used  are  a  mixture  of 
Government  and  contractor  owned  and  operated. 
Subsystem  testing  provides  an  evaluation  of  an 
armament/munition’s  subsystems,  such  as  fuze,  warhead, 
seeker,  propulsion,  etc.  Examples  of  the  types  of  testing 
conducted  on  the  various  subsystems  are: 

9.3.1.  Aerodynamic  Testing.  The  typical  aerodynamic 
test  methodologies  employed  to  determine  vehicle 
performance  include:  static/dynamic  stability  and  control 
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properties,  booster  and  shroud  separation  characteristics, 
jet  interaction  and  control  effectiveness,  inlet 
performance,  aeroheating  and  surface  pressure  distribu¬ 
tion,  and  validation  of  aerodynamic  and  aerothermal 
computations.  For  static  stability  and  control  properties, 
booster  and  shroud  separation  characteristics,  and  jet 
interaction  and  control  effectiveness  test  methodologies, 
the  measurement  of  static  force  and  moments  are  the 
critical  parameters.  The  force  and  moment  measurement 
technique  uses  a  multiple  degree-of-ffeedom  static 
balance.  Force  and  moment  data  can  be  obtained  either  as 
individual  points  in  the  flight  envelope  (pitch  pause)  or  in 
a  continuous  sweep  mode. 

9.3.2.  Aerothermodynamics.  The  aerothermal  test 
methodology  is  a  two-phase  approach;  1)  define  the 
vehicle’s  thermal  flight  environment,  and  2)  demonstrate 
material,  component,  and  structural  survivability  and 
performance.  Aerothermal  test  results  are  used  for  code 
verification  and  defining  local  heat  transfer  rates  around 
complex  geometry  such  as  control  surfaces,  window 
apertures,  and  other  protuberances.  The  test  techniques 
developed  for  material  screening  involve  placing  material 
samples  into  specially  designed  wedges  or  nosetip  holders 
and  inserting  the  test  article  into  the  flow.  Test 
techniques  for  component  and  structure  performance  and 
survivability  make  use  of  the  same  approach. 

9.3.3.  Weather/Erosion.  The  weather/erosion  test 
methodology  is  similar  to  that  of  aerothermal  in  that 
ground  testing  starts  at  the  material  level  and  progresses 
to  the  component  and  structural  level.  Two  common  test 
techniques  for  evaluating  and  comparing  material, 
component,  and  structural  response  are  the  exposure  of 
the  test  article  to  a  single  impact  or  multiple  impacts. 

9.3.4.  Aero/Electromagnetic.  Evaluation  of  the  impact 
of  the  aerothermal  environment  on  EM  performance  is  a 
relatively  new  field  that  involves  the  union  of  several 
disciplines:  aerodynamics,  aerothermal/structures,  and 
advanced  diagnostics.  Test  techniques  are  available  for 
assessing  transmission  and  possible  distortion  of  EM 
waves  (Radio  Frequency  (RF)  or  IR)  as  they  pass  through 
the  armament/munition  bow  shock,  flowfield,  and  EM 
window.  These  techniques  are  essential  for  assessing 
performance  of  seeker,  fusing,  and  communications 
systems  on  supersonic/hypersonic  munitions.  Other  test 
techniques  are  available  to  evaluate  body  and  flowfield  IR 
and  RF  signatures  which  are  essential  in  validating 
models/codes  for  the  prediction  of  flight  and  mission 
information. 

9.3.5.  Ordnance  Testing.  Test  engineers  working  in  this 
broad  discipline  support  the  program  offices  in 
specialized  areas  of  warhead  arena  testing  for  detonation 
and  coupon  tests  for  fragmentation  measurement  and 
analysis,  fuzing  function  test  for  detonating,  safe,  and  arm 
evaluations,  warhead  lethality,  and  safety  testing.  They 
are  also  involved  in  aerodynamic  and  thermodynamic 
environmental  analysis  and  tests. 


9.3.6.  Impact/Lethality.  The  general  objectives  of 
impact/lethality  tests  are  to  characterize  effects  of  high 
speed  impacts  on  materials,  components,  and  structures. 
Current  test  programs  fall  into  two  classes,  assessment  of 
the  lethality  of  kinetic  energy  armament/munitions 
requiring  impact  velocities  of  less  than  7  km/sec,  and 
assessment  of  hypervelocity  impact  events,  applicable  to 
space-based  lethality  and  orbital-debris  encounters,  at 
velocities  up  to  14  km/sec.  For  lethality  assessment, 
scaled  projectiles  and  full  and  subscale  targets  (including 
flight  hardware)  are  used.  Projectiles  which  have  been 
successfully  launched  include  standard  models  such  as 
spheres,  long-rods,  and  slugs  as  well  as  complex  models 
such  as  fragmented  projectiles,  fluid  models,  and 
segmented  rods.  For  assessment  of  hypervelocity  impact 
effects,  targets  range  from  flat  plates  to  full-scale 
hardware  such  as  satellites.  One  of  the  key  objectives  of 
both  lethality  and  hypervelocity-impact  testing  is 
assessment  of  make-up  and  propagation  rate  of  the  debris 
cloud  which  results  from  high  speed  impacts. 
Measurement  techniques  developed  include  soft-catch  of 
debris  fragments  and  use  of  witness  plates  to  examine 
damage  caused  by  debris  particles. 

9.3.7.  Structures  and  Materials  Testing.  The  structures 
and  materials  test  engineers  provide  mechanical  and 
materials  engineering  design  support  for  the  various 
munitions  programs.  Specifically,  they  are  responsible 
for  static  and  dynamic  structural  loading,  environmental 
compatibility  issues,  materials,  and  processes  selection, 
and  nondestructive  inspection  and  materials  failure 
analysis.  Finite  element  analysis  methods  utilizing  state- 
of-the-art  computer  programs  are  available  to  aid  in  the 
mechanical  design  process  and  to  resolve  vibration, 
structural  loading,  and  wear  questions.  This  discipline 
also  considers  the  effect  of  structures  and  materials  on  the 
armament/munitions  signature. 

9.3.8.  Propulsion  Testing.  Propulsion  test  engineers 
support  the  acquisition  community  throughout  a  broad 
spectrum  of  solid  rocket  and  air-breathing  engineering 
disciplines.  Some  of  these  disciplines  are  propulsion 
system  analyses  and  assessments,  propellants, 
combustion,  plume  technology,  materials,  and  design. 
They  are  also  involved  in  environmental  compatibility 
and  testing,  system  instrumentation  and  testing,  propellant 
evaluation,  and  operational  hazard  analyses. 

9.3.9.  Guidance  and  Control  Testing.  The  guidance 
and  control  test  engineers  provide  acquisition  support  in 
the  specialized  areas  of  6-DOF  guidance  computer 
simulations  (separation  analysis  and  dynamic  inflight 
simulation),  sensitivity  performance  analyses,  and  digital 
signal  processing.  Analysis  and  assessment  support  to 
program  offices  also  includes  design  studies  and 
evaluations,  sensitivity  performance,  and  support  to 
contractor  testing  programs. 

9.3.10.  Seekers  and  Sensors  Testing.  The  seekers  and 
sensors  test  engineers  provide  acquisition  support  in  such 
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specialized  areas  as  signal  processing,  IR,  RF/Millimeter 
Wave  (MMW)/Electro-Optic  (EO)/Multispectral  sensors, 
system/component  failure  analyses,  discrimination 
circuitry,  environmental  compatibility  and  testing,  and 
RCS  measurement. 

9.3.11.  Ground  and  Support  Equipment  Testing.  This 
area  involves  test  engineers  who  provide  program  support 
in  disciplines  such  as  packaging  and  transportation  of 
armament/munition  systems,  armament/munition 
container  design,  MIL-STD-1760  coordination/support, 
and  ground  test  equipment  assessment.  Analysis  and 
assessment  activities  include  container  design  and 
prototype  fabrication,  Munitions  Material  Handling 
Equipment  design  and  evaluation,  computer-aided  design 
support  for  mechanical  and  structural  design,  and 
maintenance  of  the  container  equipment  database. 

9.4.  System  Testing.  During  the  system  testing  phase 
the  various  subsystems  and  components  come  together  to 
form  the  armament/munition.  At  each  step  of  assembly 
the  component  installation  is  formally  verified  and 
validated  against  engineering  drawings.  These  are  then 
rigorously  tested  against  functional  test  procedures  for 
functionality  using  special  test  equipment.  Once 
components  are  linked,  the  subsystem’s  functionality  is 
formally  tested  again,  through  highly  controlled  and 
regulated  procedures. 

During  system  level  testing,  the  armament/munition  is 
assessed  against  the  user’s  requirements  through  both 
ground  and  flight  testing.  System  level  testing  is 
conducted  primarily  within  the  ISTFs  and  OARs.  The 
focus  of  the  testing  is  still  the  performance  of  the  various 
subsystems.  Not  until  the  next  phase,  the  total  integrated 
system  testing,  is  the  system  tested  using  fully  integrated 
or  mission  level  tasks.  System  testing  is  usually 
conducted  as  a  team  effort  by  the  Government  and  the 
contractor. 

OARs  allow  testing  under  a  controlled  environment  in 
natural  climatic  conditions.  ISTFs  and  captive  testing  are 
used  to  identify  and  resolve  problems  before  free-flight 
testing  to  assist  with  mission  planning  in  support  of  open 
air  testing,  and  to  investigate  problems  discovered  during 
open-air  events.  Separation  test  vehicles  are  used  prior  to 
live  launches  to  demonstrate  safe  separation.  Flight 
testing  begins  at  the  heart  of  the  envelope  (altitude  and 
airspeed)  and  the  heart  of  the  individual  subsystem’s 
functional  design  envelope.  Along  the  way,  as  more 
confidence  is  gained,  the  envelope  is  expanded.  Not  all 
subsystems  mature  at  the  same  rate  and  subsystem 
upgrades  occur  at  different  times.  Regression  testing  is 
often  required  when  these  upgrades  are  performed.  Many 
of  the  user’s  performance  requirements  (range,  accuracy, 
etc.)  will  be  directly  tested  during  system  testing. 
Included  in  the  system  test  phase  is  preliminary  Technical 
Order  (TO)  verification  and  validation.  These  documents 
mature  with  the  system.  Maintenance  actions  should  be 
performed  in  accordance  with  draft  TOs  and 


recommended  corrections  or  enhancements  should  be 
explored/incorporated  into  the  final  version. 

An  important  step  during  system  testing  is  the  OT&E 
certification.  This  occurs  when  the  Program  Manager 
believes  the  system  is  ready  to  transition  from  DT&E  to 
dedicated  OT&E  and  is  confident  that  the  system  will 
pass  all  required  OT&E  testing.  Details  on  this  process 
may  be  found  in  AFI  99-101  and  AFI  99-102. 

9.5.  Total  Integrated  System  Testing.  This  phase  of 
system  development  is  concerned  with  the  testing  of  the 
system  in  an  environment  which  is  as  close  to  the 
operational  environment  as  practical.  The  goal  of  this 
phase  of  testing  is  to  deliver  an  effective  and  suitable 
armament/munition  system  to  the  user.  To  accomplish 
this,  realistic  suitability  testing  must  be  performed,  in 
addition  to  realistic  effectiveness  testing.  Effectiveness 
testing  relates  directly  to  a  system’s  performance 
characteristics  and  to  mission  accomplishment. 
Suitability  testing  relates  directly  to  placing  a  system  in 
field  use  with  consideration  given  to  availability, 
reliability,  maintainability,  logistics  supportability, 
transportability,  interoperability,  compatibility,  training, 
safety,  documentation,  and  other  characteristics 
supporting  the  conduct  of  mission  level  tasks. 

Similar  to  the  previous  phase  of  testing,  total  integrated 
system  testing  is  conducted  on  OARs,  with  some  ISTF 
testing.  This  usually  requires  conducting  the  tests  at 
ranges  which  closely  approximate  operational  environ¬ 
ments.  These  ranges  will  include  threats  and  threat 
surrogates  and  operationally  representative  targets. 

9.6.  Fielded  System  Testing.  Normally,  follow-on 
OT&E  testing  will  occur  for  the  life  of  the  program.  This 
testing  is  again  primarily  conducted  in  an  open-air 
environment.  If  as  a  result  of  this  testing  a  deficiency  is 
noted,  a  major  modification  may  be  necessary.  The  level 
of  testing  for  the  major  modification  is  dependent  on  the 
degree/impacts  of  the  modification.  All  test  resource 
categories  may  be  employed  during  the  modification 
development  effort.  The  effort  usually  begins  with 
measurement  facilities  and  ends  with  OAR  testing. 
Another  type  of  testing  conducted  on  the  fielded  system  is 
the  “shelf  life”  testing  conducted  by  the  depots.  This 
testing  is  aimed  at  sensitive  components  which  may 
degrade  overtime.  Examples  include  testing  of  squibs, 
solid  fuel,  warheads,  and  similar  energetic  components,  as 
well  as  seeker,  guidance  and  control,  and  all-up-found 
tests. 

10.  Implementation  Checklist.  How  do  you  know  if 
you  have  taken  the  proper  actions  to  implement  the  T&E 
Process  for  your  test  and  evaluation  effort?  The 
following  checklist  is  provided  to  answer  this  question.  If 
you  follow  the  checklist,  a)  you  will  have  complied  with 
the  direction  in  this  manual,  and  b)  you  will  have  properly 
set  up  your  T&E  to  do  the  test.  Then  test  execution  will 
be  done  by  test  plan  and/or  handbook  containing  the  how 
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to  procedures.  If  you  get  one  or  more  negative  checklist 
answer,  you  have  more  work  to  do. 

•  Do  you  have  a  Predict-Test-Compare  Test 
philosophy? 

•  Do  test  requirements  flow  from  user/customer 
requirements? 

•  Have  you  gathered  the  appropriate  source 
documents?  (MNS,  ORD,  System  Threat 
Assessment  Report,  etc.) 

•  Have  you  gathered  or  developed  the  needed 
program  documents?  (COI,  MOE,  MOP,  TEMP, 
etc.) 


Does  your  T&E  effort  use  a  disciplined, 
scientific  process? 

Have  you  addressed  each  of  the  six  A/M  T&E 
process  steps  in  your  TEMP  and  test  plans,  and 
justified  any  tailoring  done? 

Is  the  process  being  applied  to  answer  the  COIs 
and  the  T&E  questions  in  the  TEMP? 

Does  your  T&E  effort  emphasize  use  of 
modeling  and  simulation  and  ground  tests  prior 
to  costly  flight  tests? 

Are  you  working  with  the  Munition  SFTC 
office? 

Do  the  people  on  your  T&E  effort  understand  the 
A/M  T&E  Process? 

Is  your  contractor  on  contract  to  use  and  support 
the  A/M  T&E  Process? 

Do  your  Government  T&E  agreements  require 
using  and  supporting  the  A/M  T&E  Process? 

Has  a  DSM  been  developed  and  is  it  being  used 
in  the  T&E  Process? 

Have  arrangements  been  made  to  maintain  the 
DSM  current  with  the  real  system? 

Are  your  modeling  and  simulation  efforts 
continually  updated  and  do  they  provide  constant 
feedback  for  making  improvements? 

Do  you  have  a  Test  Process  Archive  (TPA)  set 
up? 

Have  arrangements  been  made  to  keep  the  TPA 
on-going  and  accessible  throughout  the  life  cycle 
of  your  A/M  System? 

Will  the  T&E  effort  report  results  that  will  be 
used  by  decision  makers  to  support  system  life 
cycle  and  maturity  decisions? 

If  not  planning  to  use  the  A/M  T&E  Process, 
have  you  obtained  a  waiver  from  HQ  USAF/TE? 


CHAPTER  3 
Related  Topics 


11.  Single-Face-To-Customer  Office.  The  SFTC  Office 
facilitates  initial  test  planning  for  new  programs  and 
major  modification/product  improvement  programs  prior 
to  selection  of  a  RTO.  The  SFTC  manages  T&E 
investment  planning  and  maintains  the  test  process. 

11.1.  Initial  Test  Planning.  The  SFTC  typically 
provides  services  during  the  early  phases  of  new 
programs  and  in  the  early  phases  of  modification/pre¬ 
planned  product  improvement  programs.  Once  initial  test 
planning  is  completed,  the  role  of  the  SFTC  diminishes  to 
one  of  test  cognizance  and  support  of  the  RTO,  as 
requested.  The  transition  from  SFTC  to  RTO  will  take 
place  when  the  RTO  is  selected.  The  SFTC  serves  as  a 
consultant  and  works  with  customers  early  in  the  system 
development  cycle  to  develop  a  complete  understanding 


of  mission  and  system  test  requirements  and  to  assist  in 
the  use  of  a  disciplined  test  process  and  development  of 
test  documents.  The  SFTC  uses  the  test  process  to 
provide  T&E  options,  with  associated  risk,  to  the 
customer.  The  selection  of  test  resources  will  be  made  by 
the  Program  Manager  after  considering  all  available  DoD 
T&E  capabilities.  The  development  of  the  options  is  an 
iterative  process  to  ensure  that  both  the  SFTC  and  the 
customer  are  in  full  agreement  on  the  final  options 
provided. 

11.2.  Test  and  Evaluation  Resource  Investment 
Planning.  The  SFTC  offices  participate  in  early  develop¬ 
ment  test  planning  to  fully  understand  program  T&E 
resource  requirements.  Requirements  from  test  and 
logistic  centers,  program  offices,  operational  commands, 
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AFOTEC  and  any  other  agency  that  has  test  resource 
requirements  are  addressed  by  the  SFTC.  If  the  test 
requirements  exceed  current  or  planned  T&E  capabilities, 
the  SFTCs  will  advocate  those  test  investments  that  will 
meet  the  test  requirements.  The  SFTC  chairs  a  panel 
composed  of  representatives  from  the  acquisition 
community  to  validate  test  resource  requirements  from  all 
programs,  centers,  operational  commands  and  AFOTEC. 
The  panel  generates  a  prioritized  list  of  investments  and 
provides  that  list  to  the  AFMC  T&E  Operations  Panel. 
This  list  applies  to  investments  in  the  AFMC  T&E 
infrastructure  which  will  be  of  benefit  to  all  test 
customers,  not  just  the  responsibility  of  the  program 
office;  (working  through  the  appropriate  SFTC  to  the 
proper  test  center)  including  funding.  Final  decisions  on 
T&E  resource  priorities  and  their  location  are  made  by  the 
T&E  Mission  Element  Board.  The  SFTC  maintains  the 
Munitions  section  of  the  Air  Force  Test  Investment 
Strategic  Plan.  Acquisition  of  T&E  resources  is 
accomplished  by  the  individual  activity  or  by  their 
acquisition  agent.  The  SFTC  chairs  the  Reliance  Air 
Armament  T&E  Resource  Committee  for  air-to-surface, 
air-to-air,  and  surface-to-air  armament/munition  systems. 
This  is  a  tri-service  activity  aimed  at  eliminating 
duplication  of  service  T&E  investments.  For  more 
information  on  the  test  investment  process,  reference  AFI 
99-109,  Test  Resource  Planning. 

11.3.  Armament/Munitions  Test  and  Evaluation 
Process.  The  SFTC  documents  and  advocates  the  T&E 
process.  It  works  to  keep  T&E  capability  current  with 
technology  and  system  requirements.  The  SFTC 
periodically  hosts  conferences  to  expand  test  process  use 
and  facilitate  its  implementation. 

12.  Aircraft-Stores  Certification  Program  (SEEK 
EAGLE). 

12.1.  Purpose.  The  purpose  of  the  Air  Force  SE 
program  is  to  certify  aircraft-store  configurations  on 
production  or  developmental  aircraft  (fixed  or  rotary 
wing)  to  meet  operational  requirements  specified  by  the 
using  command.  The  program,  governed  by  AFI  63-104, 
establishes  a  standard  Air  Force  process  for  all  aircraft- 
store  certification  activities.  Production  or  developmental 
stores  include  munitions,  missiles,  bombs,  rockets,  mines, 
torpedoes,  dispensers,  detachable  fuel  and  spray  tanks, 
pods  (electronic  countermeasures,  instrumentation, 
camera,  gun,  refueling,  thrust  augmentation  etc.),  targets, 
chaff  and  flares  and  suspension  equipment  (adapters, 
racks  and  pylons).  Certification  provides  the  using 
command  the  capability,  including  the  applicable  aircraft 
TOs  and  OFPs,  to  upload,  download,  safe  carriage  and 
separation  and  verify  ballistics  accuracy  of  stores  on 
aircraft.  Certification  includes  development  of  Combat 
Weapons  Delivery  Software,  which  provides  an 
automated  mission  planning  capability  for  ballistic  and 


safe  escape  data,  and  Combat  Stores  Loading  Software, 
which  provides  automated  aircraft-stores  limitations. 

12.2.  Process.  AFI  63-104  defines  the  SE  process  and 
the  SE  Engineering/Test  Capabilities  Handbook  (August 
1992)  and  identifies  the  location,  primary  mission,  and 
major  aircraft-store  certification  test  resources  available  at 
various  government  test  and  analysis  facilities.  The  SE 
Process  (figure  8)  includes  the  engineering  analysis, 
computer  simulations,  WTTs,  ground  tests  and  flight  tests 
to  obtain  the  engineering  data  needed  to  update  the 
aircraft  TOs  and  OFP  and  to  verify  OFP  ballistics 
accuracy.  The  SE  process  begins  with  a  SE  Request 
(SER)  submitted  by  the  using  command  or  the  Directorate 
of  International  Programs  (SAF/IAY)  for  Foreign  Military 
Sales/Intemational  programs  and  ends  with  the 
publication  of  related  TOs  and  user  acceptance  of  the 
ballistic  armament/munition  accuracy  in  the  OFPs,  as 
shown  in  figure  8.  Program  Management  Directive  5077 
provides  direction  and  funding  for  certification  projects. 
In  developmental  or  major  modification  to  inventory 
aircraft-stores,  the  using  command  or  SAF/IAY  submits  a 
SER  in  parallel  with  the  ORD.  Each  SER  is  made  into 
one  or  more  projects  designed  to  include  activities 
required  to  complete  the  certification  process.  Each 
project  is  tailored  from  a  standard  template  by  the 
requirements,  performance,  schedule,  cost  and 
constraints.  Offices  of  Primary  Responsibility  (OPRs)  are 
identified  in  AFI  63-104.  An  OPR  is  identified  with 
activity  duration,  start,  completion,  funding  (planned  and 
actual),  for  each  project  activity  in  the  SE  Management 
Support  System  (SEMSS). 

12.3.  SEEK  EAGLE  and  the  Systems  Acquisition 
Process.  To  reduce  the  time  to  obtain  a  fully  operational, 
certified  aircraft-store  configuration,  SE  will  be  integrated 
with  acquisition  programs  from  the  beginning.  Program 
offices  begin  SE  planning  not  later  than  the  Dem/Val 
phase  in  both  aircraft  and  store  programs.  Certification  of 
the  baseline  aircraft- store  configurations  defined  by  the 
using  command  in  the  SER  is  completed  applying 
Integrated  Weapon  System  Management  guidelines. 
Follow-on  certifications  are  accomplished  by  the  Air 
Force  SE  Office  (AFSEO)  by  the  completion  date 
specified  in  the  SEMSS. 

12.4.  SEEK  EAGLE  and  Aircraft-Stores  Integration. 

Aircraft-stores  integration  includes  avionics,  electrical 
and  mechanical  integration  or  associated  modification  to 
the  aircraft  or  store  to  provide  operational  interfaces 
between  the  aircraft  and  store.  SE  does  not  conduct  this 
integration,  nor  the  development  or  modification  of  the 
aircraft  or  store  to  achieve  the  aircraft-store 
configurations  that  are  certified  under  SE.  Integration  and 
modification  activities  are  the  responsibility  of  the  store 
and  aircraft  System  Program  Offices.  The  Weapon 
System  Review  process  and  the  Weapon  Integration  Plan 
(WIP)  are  established  to  accomplish  seamless 
armament/munition  system  management.  A  WIP  is  an 
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agreement  between  the  aircraft  and  store  program  offices 
and  outlines  objectives,  procedures,  responsible  agencies, 
and  resource  requirements  for  integrating  an 
armament/munition  onto  an  aircraft.  SE  requirements 
must  be  defined  early  in  the  integration  or  modification 
efforts  to  help  define  the  program,  to  ensure  realistic 


operational  configurations  are  considered  early  in  the 
program  and  to  integrate  SE  to  the  maximum  extent 
possible,  into  DT&E  and  IOT&E.  The  AFSEO  will 
ensure  SE  requirements  are  properly  completed  and  used 
as  a  database  to  meet  future  aircraft-store  certifications. 


Figure  8.  SEEK  EAGLE  Process. 


12.5.  Production  Munitions  Requirements,  Fore¬ 
casting  and  Budgeting.  Inventory  armament/muni-tions 
requirements  are  forecasted  annually.  Many  munitions 
and  missiles  are  out  of  production.  When  there  is  a 
requirement  in  a  SER  for  a  store  that  is  out  of  production, 
the  SE  program  element  will  plan  and  budget  for  it.  If 
there  is  no  SER,  it  is  the  responsibility  of  the  aircraft-store 
program  office  to  plan  and  budget  for  the  required 
inventory  item. 

12.6.  Flight  Clearance  (FC).  The  using  commands 
submit  SERs  requesting  a  FC  to  support  a  limited 
duration  operational  need  or  an  operational  flight  test. 
The  product  is  a  FC  authorization  letter/message.  Flight 
clearances  should  only  be  requested  when  mission 
accomplishment  will  be  impacted,  because  they  are 
supported  at  the  expense  of  routine  certification  efforts. 


13.  Test  Resource  List.  Test  facilities  and  capabilities 
must  accommodate  the  mix  of  systems  in  the  various 
phases  of  their  life  cycle  as  well  as  the  mix  in  the  level  of 
interaction  of  the  systems/subsystems.  The  functional 
interrelationships  among  munitions,  EC  and 
aircraft/avionics  indicated  in  figure  9  must  also  be 
reflected  in  test  facilities  and  capabilities.  Figure  10 
depicts  the  use  of  test  facilities  and  capabilities  across 
these  three  areas.  In  this  figure,  test  resources  are  aligned 
with  the  six  T&E  process  categories  to  show  which  part 
of  the  T&E  process  they  primarily  support.  Figure  11 
shows  the  location  of  Armament/Munitions  Test 
Facilities. 


Figure  9.  Test  and  Evaluation  Resource  Overlap. 
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Figure  10.  Armament/Munition  Test  and  Evaluation  Resources. 
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Figure  11.  Location  of  Armament/Munition  Test  Facilities. 
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13.1.  Digital  Models  and  Computer  Simulation. 

13.1.1.  Munitions  Scientific  Computational 
Capabilities.  These  capabilities  are  housed  in  the 
Freeman  Computer  Science  Center,  AFDTC,  Eglin  AFB, 
FL  and  the  Central  Computer  Facility  at  Arnold  AFB, 
TN.  They  use  computer  capabilities  for  modeling  and 
simulation  of  armament/munitions  in  support  of 
laboratory  research,  engineering  development  and  test. 
Modeling  and  simulation  up  to  and  including  6-DOF 
models  and  high  fidelity  seeker  models  can  be 
accomplished.  In  addition  to  the  support  of  development, 
models  and  simulations  are  used  to  support  the  safety 
footprint  and  other  aspects  of  test  planning  and  analysis 
including  armament/  munition  fragmentation  and  lethality 
evaluation  models.  CFD  capability  is  also  available  to 
investigate  flowfields  around  munitions  and 
aircraft/munitions  configurations. 

13.1.2.  Aerodynamic  Scientific  Computational 
Capabilities.  For  armament/munitions  support,  this  is 
mainly  the  CFD  to  model  airflow  around 
armament/munitions  as  they  are  carried  on  aircraft.  This 
is  a  complementary  capability  to  wind  tunnel  tests  to 
predict  extremely  complex  configurations.  These 
capabilities  are  located  at  the  Arnold  Engineering 
Development  Center  (AEDC),  Arnold  AFB,  TN  and 
AFDTC. 

13.2.  Measurement  Facilities. 

13.2.1.  Multispectral  Signature  Measurement 
Systems.  These  systems  are  a  combination  of  ground  and 
airborne  instrumentation  systems  to  collect  signature 
measurements  on  targets,  countermeasures,  and 
backgrounds.  The  instrumentation  covers  the  EM 
spectrum  from  ultraviolet  (UV)  through  IR  and  the 
appropriate  portions  of  MMW.  Data  is  collected  and 
presented  in  the  radiometric,  spatial,  and  spectral 
domains.  In  addition,  missile  data  (track  point  and 
performance)  are  gathered.  Airborne  systems  are 
available  on  various  platforms  (F-15,  UH-1,  C-130)  for 
signature  measurements  in  UV,  IR  and  MMW  of  aerial 
and  surface  targets,  countermeasures  and  backgrounds, 
both  day  and  night,  with  the  capability  of  evaluating  air- 
to-air,  air-to-surface,  surface-to-surface,  and  surface-to-air 
seeker  performance  in  a  captive  carry  scenario. 

Ground  based  systems  covering  UV,  IR  and  MMW  are 
available  for  signature  measurements  of  aerial  and  surface 
targets,  countermeasures  and  backgrounds.  Threat 
vehicle  signature  production  and  assessment  using  a  fleet 
of  over  50  threat  combat  systems  and  associated 
countermeasures  are  also  available.  These  signatures  are 
processed  and  stored  in  the  Target  and  Background 
Information  Library  System.  Evaluation  of  surface-to-air, 
air-to-air  seeker,  and  tower/tumtable  performance  against 
targets  and  countermeasures  is  also  provided. 

13.2.2.  Store  Mass  and  Physical  Properties  Measure¬ 
ment  Systems.  These  systems  are  used  to  determine 


weight,  center  of  gravity,  and  moments  of  inertia  of 
armament/munitions  and  shapes/stores/pods  carried 
and/or  released  from  aircraft.  Measured  values  are  used 
to  determine  the  weight  and  balance  of  the  aircraft  as  well 
as  store  loads,  flutter,  performance,  stability  and  control 
and  separation/post  release  analysis  (e.g.,  for  store 
ballistics  trajectory). 

13.2.3.  Sled  Test  Tracks.  Sled  test  tracks  have  the 
capability  to  test  captive,  launch  into  free  flight,  and 
impact  test  under  accurately  programmed,  closely 
controlled,  and  rigorously  monitored  conditions.  Captive 
testing  can  be  used  to  study  rain  erosion,  aerothermal  or 
particle  impact  effects  on  radomes,  sensor  windows,  fins 
and  other  munitions  components.  Sensor  cover  removal 
and  skin  cutting  for  submunition  deployment  can  also  be 
evaluated.  Launch  into  free  flight  can  be  accomplished 
using  actual  aircraft  hardware.  Armament/munition 
trajectory,  flight  control  information  and  inertial  sensor 
performance  are  examples  of  the  data  that  can  be 
obtained.  Impact  tests  are  conducted  with  targets 
suspended  above  the  rails  or  beyond  the  end  of  the  rails  to 
assess  fuzing  system  and  penetrator  performance. 

13.2.4.  Wind  Tunnels,  Arc  Tunnels,  and  Propulsion 
Cells.  The  wind  tunnels,  arc  tunnels,  and  propulsion  cells 
are  located  at  AEDC,  Arnold  AFB,  TN.  These  facilities 
provide  a  capability  to  support  the  design,  development 
and  improvement  of  armament/munitions.  The  tunnels 
are  equipped  with  moveable  supports  on  which 
armament/  munition  models  are  mounted  to  collect  data 
as  if  the  armament/munition  was  in  free  flight.  The 
support  is  moved  to  simulate  changes  in  flight  maneuvers. 

13.2.4.1  Wind  Tunnels.  The  wind  tunnels  are  used  to 
investigate: 

•  Aerodynamic  forces  when  munitions  are  carried 
or  separated  from  aircraft  and  during  free  flight 
to  the  target. 

•  Aerodynamic  performance  of  inlets  for  air 
breathing  propulsion  of  armament/munitions. 

•  Complex  aerodynamic  interactions  from 
controls,  jets,  protuberances,  etc. 

•  Aerothermal  environment  determination  used  to 
assess  heat  load  in  supersonic/hypersonic  free 
flight. 

•  Booster/shround  separation  effects. 

•  Optical  and  RF  wave  distortion  by  flowfields 
surrounding  the  armament/munition. 

•  Thermostructural  performance  in  free  flight  with 
and  without  the  effects  of  erosive  weather 
environments. 

13.2.4.2  Arc  Tunnels.  The  arc  tunnels  are  used 
primarily  to  assess  thermostructural  performance  of 
components  and  full-up  armament/munitions  in  high- 
supersonic  and  hypersonic  free  flight.  Specifically,  the 
arc  tunnels  are  used  to: 
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•  Assess  material  sample  thermostructural 
performance  in  high-speed  heating 
environments. 

•  Validate  structural  integrity  of  prototype 
components  and  armament/munitions  under 
severe  aerothermal  and  dust-erosion  conditions. 

•  Investigate  effects  of  material  thermal  response 
and  ablation  on  transmission  of  RF  signals 
through  EM  windows  on  high-speed 
armament/munitions. 

13.2.4.3.  Propulsion  Cells.  The  propulsion  cells  are 
used  to  test  air  breathing  and  rocket  propulsion  systems 
performance.  Specifically,  propulsion  cells  are  used  to: 

•  Assess  thermostructural  integrity  of  engine 
components  and  full-up  prototypes  under 
realistic  flight  conditions. 

•  Assess  performance  in  terms  of  thrust,  fuel 
consumption,  pressure  loss,  etc. 

•  Identify  critical  failure  mechanisms  in  a 
controlled  environment  which  does  not  put 
personnel  or  costly  assets  at  risk. 

•  Assess  performance  of  the  combined  engine- 
airframe  assembly  for  full-up 
armament/munitions. 

13.2.5.  Ballistic  Ranges.  Arnold  Engineering  Develop¬ 
ment  Center,  Arnold  AFB  TN  has  several  ballistic  ranges 
that  support  armament/munitions  testing.  In  ballistic 
ranges,  projectiles  are  fired  from  either  conventional 
powder  guns  or  two-stage,  light-gas  guns  down  a  range, 
usually  in  a  controlled  environment,  in  order  to  make 
measurements  in  free  flight  or  during  impact  at  the  end  of 
the  flight.  Ballistic  ranges  are  used  to: 

•  Assess  impact  phenomenology  to  determine 
armament/munition  lethality  and  effectiveness. 

•  Evaluate  armament/munition  static  and  dynamic 
stability  using  subscale  models  in  free  flight. 

•  Investigate  effects  of  the  plasma  of  ionized  air 
surrounding  armament/munitions  in  hypersonic 
flight  or  transmission  of  RF  signals  to  determine 
interference  in  seeker,  fusing,  and 
communications  systems  performance. 

•  Evaluate  thermostructural  effects  of  material 
samples  exposed  to  severe  aeroheating,  rain, 
snow,  ice,  and  dust  environments. 

•  Evaluate  acceleration  effects  on  component 
performance. 

13.2.6.  Central  Inertial  Guidance  Test  Facility.  This 
facility  has  the  capability  to  test  GPS,  inertial  systems, 
and  their  components  (gyroscopes  and  accelerometers)  for 
armament/munitions.  Capabilities  include  precision  rate 
tables,  centrifuges,  a  combined  environmental  test 
chamber  (altitude,  temperature,  humidity,  and  vibration), 
dynamic  flights,  jamming  and  spoofing,  and  the  full 
gamut  of  field  testing  using  vehicles  (van),  rocket  sleds, 
and  aircraft. 


13.2.7.  Radar  Target  Scatter  Facility  (RATSCAT)/ 
RATSCAT  Advanced  Measurement  System  (RAMS). 
RATSCAT  is  the  DoD  center  of  expertise  for  monostatic 
and  bistatic  RCS  measurement  of  aircraft  (with  and 
without  armament/munitions),  spacecraft,  unmanned 
vehicles,  and  decoys.  The  RAMS  is  used  for  performing 
RCS  measurements  on  very  low  observable  test  articles. 
For  armament/munition  applications,  these  measurements 
can  be  on  the  armament/  munition  itself  to  determine  its 
RCS,  or  for  the  armament/munition  contribution  to  the 
RCS  of  the  carrier  aircraft  with  the  armament/munition  in 
its  carriage  configuration. 

13.2.8.  Gun  Test  Facilities.  These  facilities  conduct 
instrumented  ground  and  airborne  gun  tests.  Three  test 
areas  within  the  Eglin  complex  each  have  a  permanently 
installed  and  instrumented  GAU-8/A  30-mm  gun  system 
and  a  single-shot  30-mm  Mann  barrel.  Other  gun  systems 
can  also  be  mounted  for  specific  tests.  Some  of  the  types 
of  tests  conducted  are  gun  and  ammunition  performance, 
depleted  uranium  projectile,  armor  plate  penetration, 
projectile  and  fuze  characterization,  target  and  component 
vulnerability  studies,  and  terminal  ballistic  studies. 
Typical  data  collected  are  muzzle  velocity,  chamber 
pressure,  relative  action  time,  barrel  temperature, 
meteorological  conditions,  projectile  yaw,  and  dispersion. 

13.2.9.  Arena  Test  Facility.  Arena  tests  are  conducted 
to  characterize  the  fragmentation  pattern  of  a  warhead. 
Data  collected  are  used  to  determine  warhead  lethality 
against  threat  targets.  Typically,  warhead  fragment  speed, 
size,  and  distribution  data  are  collected.  The  warhead  is 
placed  in  the  center  of  an  “arena”  and  detonated. 
Cameras,  witness  plates,  photography,  blast  gages, 
fragment  collection  devices  ,  etc.  measure  the  attributes  of 
the  warhead.  This  raw  data  are  used  in  models  to 
determine  warhead  lethality. 

13.2.10.  Fuze  Test  Facility.  The  Fuze  Test  Facility  is 
the  primary  munitions  integration  laboratory.  This 
facility  is  an  instrumented  laboratory  capable  of 
performing  research,  development,  and  engineering 
service  tests  on  fuzes  and  associated  ordnance  devices. 
Simulated  environments  are  provided  to  determine 
whether  fuzes  and  associated  devices  are  safe  and  will 
function  as  intended.  Some  of  the  capabilities  include 
vibration,  shock,  jolt/jumble,  salt/fog,  sand/dust,  x-ray, 
static  drop,  and  failure  analysis/teardown. 

13.3.  Integration  Laboratories. 

13.3.1.  Air-to-Air  Missile  Buildup  Facility.  The  Air-to- 
Air  Missile  Buildup  Facility  has  two  secure  vaults 
dedicated  to  air-to-air  missile  test  projects;  analysts  for 
AIM-7,  AIM-120,  ASRAAM,  AIM-9,  and  AIM-9X; 
secure  workstations  for  data  reduction;  a  library  of 
software  codes,  research  references,  and  intelligence 
documentation;  3-  and  6-DOF  simulation  capabilities;  and 
a  fully  automated  system  test  facility  for  AIM- 120  missile 
preflight  and  integration. 
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13.3.2.  Air-to-Surface  Guided  Weapon  Buildup 
Facility.  The  Air-to-Surface  Guided  Weapon  Buildup 
Facility  is  used  for  the  assembly,  final  integration  testing, 
and  inspection  of  guided  weapons  prior  to  flight  test.  The 
facility  integrates  fuzes,  warheads,  actuators,  data  links, 
inertial  measurement  unit/GPS,  propulsion, 
airframes/structures,  hardware/software,  guidance/control, 
weapon  sensors,  and  flight  termination  systems.  Buildup 
and  modifications  of  bombs/missiles,  as  well  as  integrated 
systems  tests,  are  performed  at  this  facility.  The  test 
chambers  are  rated  for  Category  1  explosives. 

13.4.  Hardware-in-the-Loop  Facilities.  The  Guided 
Weapons  Evaluation  Facility  (GWEF)  is  the  primary 
munitions  HITL  facility.  This  facility  uses  a  combination 
of  actual  armament/munitions  hardware  components  and 
target  background  simulators  to  conduct  HITL  testing  to 
evaluate  armament/munition  performance  throughout  the 
trajectory  from  launch  to  target  intercept/impact.  T&E  of 
precision  guided  armament/munitions  using  IR,  RF,  laser, 
MMW,  EO,  multispectral,  and  midcourse  inertial 
techniques  are  conducted.  Injection  of  countermeasures 
and  counter-countermeasure  capabilities  are  also  possible. 
The  use  of  this  facility  allows  much  more  data  to  be 
gathered  since  testing  is  ground  based,  lower  cost,  and 
nondestructive  as  opposed  to  flight  testing.  In  addition, 
this  capability  allows  many  “what  ifs”  to  be  run  and 
therefore  focuses  the  more  expensive  flight  testing  on 
only  the  most  critical  areas  that  require  flight  test 
validation. 

13.5.  Installed  System  Test  Facilities. 

13.5.1.  Preflight  Integration  of  Munitions  and  Elec¬ 
tronic  Systems  (PRIMES).  This  facility  consists  of  a 
fighter-sized  anechoic  chamber  and  six  shielded 
laboratories  providing  secure,  realistic  testing  in  a 
controlled  RF  environment  to  support  one-on-one  or 
many-on-one  tests  in  static  or  dynamic  flight  simulation 
conditions.  Bomber  sized  aircraft  can  be  placed  outside 
PRIMES  and  linked  to  the  facility.  The  laboratories 
include  analog  and  digital  equipment  to  simulate  EM 
flight  test  environments.  Armament/munition  guidance 
commands,  auto-pilot  functioning,  and  seeker 
performance  can  be  monitored  during  tests.  Using 
PRIMES,  many  system  problems  associated  with  early 
phases  of  development  can  be  identified  prior  to  the  start 
of  flight  testing,  and  therefore  reduce  flight  test  costs  and 
allow  the  more  productive  use  of  available  aircraft. 

13.5.2.  Linked  Facilities.  Total  aircraft-avionics- 
munitions  system  testing  can  be  conducted  using  a  fiber 
optic  link  between  the  GWEF  and  the  PRIMES.  The 
aircraft  can  be  located  in  the  anechoic  chamber  in  the 
PRIMES  and  the  armament/munition  in  the  GWEF.  The 
real-time  fiber  optic  link  transfers  all  information 
normally  transmitted  from  the  aircraft  to  the  armament/ 
munition.  This  allows  closed  loop,  end-to-end  ground  test 
of  the  total  weapon  system  (aircraft/EC/munitions) 


simulating  prelaunch,  launch  and  post  launch  phases  of  an 
armament/munition  employment. 

13.5.3.  Climatic  Facility.  This  facility  has  test  chambers 
to  conduct  tests  on  armament/  munitions  and  their 
components.  These  chambers  can  duplicate  various 
conditions  including  sun,  wind,  rain,  dust,  combined 
conditions  or  all  weather  (arctic  to  jungle-rain,  wind, 
snow),  temperature-altitude,  and  salt.  The  main  chamber 
can  conduct  tests  on  the  armament/munitions  loaded  on 
launch  platforms. 

13.6.  Open  Air  Ranges. 

13.6.1.  Armament  Systems  Test  Environment 
(ASTE)/  Gulf  Test  Range  (GTR).  The  ASTE  consists 
of  724  square  miles  of  varied,  multi-environmental  land 
area  with  45  test  areas,  34  test  systems/facilities  and  26 
multipurpose  systems/facilities  (instrumentation,  data 
transfer,  communication,  mission  control,  targets — fixed, 
mobile,  remote  controlled,  and  scoring)  for  T&E  of 
armament/munitions.  The  major  test  areas  are  Air-to- 
Surface  Test  Ranges,  Gun  Test  Facilities,  Electro- 
optical/MMW  Evaluation  sites,  and  Static  Warhead  Test 
Arenas  (unitary  blast/fragmentation,  explosively  formed 
projectile,  shaped  charge).  The  ASTE  also  includes 
special  targets  ranging  from  simulated  Warsaw  Pact  A/C 
shelters  and  runways  to  remote  controlled  tanks;  and  test 
facilities  that  are  one-of-a-kind  in  the  world;  e.g.,  the 
shallow  water  mine  countermeasure  testing  facility  and 
the  HELLFIRE  Integration  Facility  and  Test  Range.  The 
GTR  encompasses  86,500  square  miles  of  the  Gulf  of 
Mexico.  This  area  is  used  for  long-range,  all-altitude  air- 
to-air  (many  versus  many),  long  range  air-to-surface, 
surface-to-air/drone  target  engagements,  and  long-range 
anti-ship  air-to-surface  and  surface-to-surface  armament/ 
munitions  evaluations.  ASTE  and  GTR  are  adjacent  and 
provide  an  extensive  land/sea  test  capability,  with  a  total 
airspace  of  89,062  square  miles,  that  is  unique  within  the 
Air  Force.  Further,  the  tri-service  Southeast  Test  and 
Training  Area  can  provide  over  108,000  square  miles  of 
airspace  (with  varying  altitude  restrictions).  An  overland 
supersonic  corridor  is  10  nautical  miles  (NM)  from  Eglin 
AFB  and  2  over  water  supersonic  areas  are  in  the  GTR. 
A  fleet  of  primarily  F-15  and  F-16  aircraft  with  standard 
programmable  digital  instrumentation  and  the  capability 
for  unique  project  modification  is  available.  Other 
aircraft  types  are  available  or  can  be  obtained  on  loan,  if 
required.  Aircraft  can  be  deployed  to  virtually  any 
location  to  conduct/support  testing.  A  general 
instrumentation  pod  is  used  on  different  types  of  aircraft 
with  minimum  aircraft  modification.  The  pod  includes  an 
instrumentation  recorder,  video  time  inserter,  video 
recorder,  time  code  generator,  data  acquisition  system, 
data  encryptor,  and  telemetry  transmitter.  Airborne 
targets  are  a  mix  of  subscale  and  full-scale  drones  used  in 
conjunction  with  air-to-air  missile  tests  over  the  GTR 
along  with  the  command,  control,  and  tracking  system  to 
support  the  tests.  Simultaneous  control  of  up  to  six 
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drones  flying  individually  or  in  realistic  tactical 
formations  at  all  altitudes  is  possible.  The  system  will 
provide  Time,  Space,  Position  Information  (TSPI)  on  four 
drones,  four  shooter  aircraft,  and  four  air-to-air  missiles. 
Selective  flight  termination  of  drones  or  missiles  is 
included  in  the  system.  GTR  supports  over-the-horizon 
missions  using  E-9  aircraft.  Approximately  1000  mobile 
ground  targets  are  available,  including  actual,  simulated, 
or  surrogate  vehicles.  Remote  control  of  six  vehicles 
simultaneously  is  possible.  Over  1000  fixed  targets 
(bunkers,  runways,  railroads,  vehicles,  etc.),  are  also 
available.  Sea  targets  are  acquired  for  specific  tests  with 
remote  control  requirements. 

13.6.2.  Electromagnetic  Test  Environment  (EMTE). 
The  collocation  of  the  EMTE  with  the  extensive 
munitions  testing  capabilities  at  AFDTC  allows  for 
testing  armament/munitions  systems  in  a  realistic  RF 
environment.  Instrumentation  on  EMTE  supports 
precision  tracking  analysis  with  TSPI  sources, 
multispectral  measurements  and  analysis,  and  dynamic 
RCS  and  target  signature  measurement.  The  capability  to 
accomplish  extensive  integration  testing  at  both  the 
component  and  system  levels  is  available. 

13.6.3.  Utah  Test  and  Training  Range  (UTTR).  The 
UTTR  encompasses  8,125  sq  NM  of  restricted  airspace 
which  can  be  expanded  to  17,000  sq  NM  through  adjacent 
Military  Operating  Areas  (MOAs)  (in  an  area  207  by  92 
NM).  Land  space  is  2,700  sq  NM  of  DoD  land  and 
14,300  sq  NM  of  Bureau  of  Land  Management,  State  of 
Utah,  and  a  small  amount  of  privately  owned  lands 
underlying  the  restricted  air  space  and  MOAs.  This  large 
overland  airspace  and  ground  space  allow  for  large  safety 
footprints  and  long  trajectory  legs  required  by  Precision 
Guided  Munitions  (PGMs),  smart  munitions  (such  as  off 
axis  High  Speed  Anti-Radiation  Missile  launches)  and 
cruise  missiles.  Major  munitions  test  areas  include  12 
targets  for  testing  conventional  munitions,  fuzes,  high 
volume  propagation  detonations  (demonstrated  500,000 
pounds),  and  laser  guided  armament/munitions;  four 
highly  instrumented  targets  used  for  testing  of  PGMs, 
smart  armament/munitions,  and  home  on  emitter  seeking 
missiles;  two  cruise  missile  impact  targets;  and  five  air-to- 
surface  tactical  target  complexes.  All  UTTR  test  areas  are 
capable  of  munitions  tracking,  data  collection  and 
transfer,  telemetry  acquisition  and  recording, 
communications,  mission  control,  and  full  data  reduction. 
The  four  highly  instrumented  targets  are  capable  of  45 
NM  armament/munition  trajectories,  remote  command/ 
control  targets  and  range  instrumentation,  and 
armament/munition  deliveries  using  realistic  scenarios. 
The  low  population  density  surrounding  the  range  and 
low  density  of  high  value/manned  sites  within  the  range 
safety  footprint  allows  for  maximum  flexibility  in 
planning  test  mission  trajectories. 


14.  Armament/Munition  Effectiveness  Testing. 
Scientifically  validated  data  are  required  by  the  Joint 
Technical  Coordinating  Group  for  Munitions 
Effectiveness  (JTCG/ME)  to  produce  Joint  Munitions 
Effectiveness  Manuals  (JMEMs).  The  JMEMs  are  used 
by  operational  mission  planners  to  select  which 
armament/munition  and  how  many  are  required  to 
achieve  the  required  level  of  damage  on  a  given  target. 
The  test  program  provides  the  data  for  these  manuals. 
These  data  include  physical  and  functional  characteristics 
of  the  armament/munition.  The  data  are  placed  in  a 
computer  model  to  assess  the  viability  of  a  new  system  or 
new  tactics.  Factors  that  must  be  fully  considered  to 
determine  operational  effectiveness  of  an 
armament/munition  include: 

•  Target  Vulnerability.  An  assessment  of  how 
vulnerable  a  target  is  to  specific 
armament/munition  damage  mechanisms. 

•  Armament/Munition  Characteristics. 
Quantification  of  damage  producing  mechanisms 
and  reliability  of  the  armament/munition. 

•  Delivery  Accuracy.  A  measure  of  a 
armament/munition  system’s  ability  to  place 
armament/  munitions  on  a  target. 

14.1.  Target  Vulnerability.  Detailed  information  about 
the  target  is  required  to  understand  the  effectiveness  of  an 
armament/munition.  Once  this  information  is  known,  an 
analyst  can  assess  the  ability  of  known 
armament/munition  characteristic  damage  mechanisms  to 
achieve  the  desired  results.  A  computer  model  of  the 
target  is  developed  and  damage  mechanism  data,  obtained 
during  testing,  is  applied  against  the  target. 

14.2.  Armament/Munition  Characteristics.  Knowing 
the  kind  of  damage  a  warhead  can  produce  is  required  to 
determine  the  amount  of  damage  required  to  defeat  a 
specific  target.  The  JTCG/ME  has  developed 
standardized  testing  procedures  to  facilitate  a  meaningful 
evaluation  of  new  armament/munitions.  The  following 
damage  mechanism  data  will  be  collected: 

•  Blast 

•  Fragmentation 

•  T  arget  penetration 

•  Combined  damage  mechanisms 

•  Fire 

These  data  are  also  used  in  safe  escape  analysis  to  prevent 
damage  to  the  delivery  aircraft  from  armament/munitions 
fragmentation. 

14.3.  Delivery  Accuracy.  Delivery  accuracy  is  a 
measure  of  a  armament/munition  system’s  ability  to  place 
armament/munitions  on  a  target.  The  JTCG/ME  sponsors 
accuracy  tests  on  instrumented  ranges.  The  data  obtained 
during  these  tests  are  used  to  create  and  update  computer 
models  that  generate  delivery  accuracy  estimates. 

15.  Targets.  Armament/munition  test  programs  require 
targets,  target  signatures,  and  threat  signatures.  Targets 


AFMAN  99-104  1  August  1995 


29 


are  often  surrogates  for  real  threats  and  should  replicate 
the  threat  performance  and  signature  to  the  maximum 
extent  practical.  Some  surface  targets  have  a  high  degree 
of  threat  replication  and  are  called  ’’threat  simulators."  A 
threat  simulator  is  defined  to  be  a  target  that  matches  the 
threat  so  well  that  the  same  armament/munition  system 
statistical  performance  will  be  obtained  for  the  threat  and 
the  threat  simulator.  Other  targets,  particularly  aerial 
targets,  do  not  have  the  threat  replication  of  a  threat 
simulator,  and  are  called  "targets."  These  targets  require 
the  use  of  a  variety  of  analytical  methods  to  extrapolate 
performance  from  that  observed  using  the  substitute  target 
to  the  performance  expected  against  threats.  The 
extrapolation  process  depends  on  the  availability  of 
adequate  threat  intelligence  data.  The  Defense 


Intelligence  Agency  (DIA)  is  responsible  for  providing 
these  data.  The  AFMC  Center  Intelligence  Office  will 
work  with  the  System  Program  Office  and  the  RTO  to 
obtain  the  most  current/detailed  threat  information  and  to 
prepare  a  DIA  Intelligence  Production  Requirement 
document  (DD-1497)  to  request  threat  data  required  to 
accomplish  the  test  program. 

15.1.  Joint  Service  Target  Responsibilities.  Various 
types  of  armament/munitions  are  developed  to  engage 
many  types  of  aerial,  surface,  and  space  threats.  The 
Service  Acquisition  Executives  have  defined  service 
responsibilities  for  providing  test  targets  and  target  data  to 
satisfy  test  requirements.  These  responsibilities  are 
shown  in  table  8. 


Table  8.  Targets  Points  of  Contact. 

TYPE  OF  TARGET 

RESPONSIBLE  SERVICE 

POINT  OF  CONTACT 

Full-Scale  Fixed 

Wing  Aerial  Targets 

Air  Force 

AFDTC/DR 

SFTC  Office 

Eglin  AFB.FL  32543 

Subscale  Fixed  Wing 
Aerial  Target 

Navy 

Naval  Air  Warfare  Center  (NAWC)AVeapons  Division 
Point  Mugu,  CA  93042 

Rotary  Wing  Aerial 
Target 

Army 

PMITTS/TMO 

Huntsville,  AL  35898 

Ground  Targets 

Army 

PMITTS/TMO 

Huntsville,  AL  35898 

Maritime  Targets 

Navy 

NAWC/Weapons  Division 

Point  Mugu,  CA  93042 

Space  Targets 

Ballistic  Missile  Defense 
Organization  (BMDO) 

BMDO 

Washington,  DC  20301 

15.2.  Threat  Simulators  and  Targets.  Some  surface 
targets  have  sufficient  threat  replication  to  be  designated 
as  threat  simulators.  A  process  used  to  VV&A  a  threat 
simulator  is  available  from  the  SFTC  office.  The 
measured  signature  of  a  threat  substitute  is  compared  to  a 
threat  signature  which  is  either  measured  or  calculated 
using  best  available  DIA  intelligence  data.  Engagement 
modeling  and  HITL  simulation  may  be  used  in  threat 
simulation  VV&A.  For  targets  that  do  not  have  adequate 
threat  replication  to  be  validated  as  a  threat  simulator  (this 
is  the  case  for  all  aerial  targets  in  the  terminal  phase  of  a 
test),  available  modeling  and  simulation  techniques, 
including  signature  modeling,  engagement  modeling, 
HITL  simulation,  and  captive  flight  testing,  must  be  used 
to  extrapolate  test  data  obtained  against  the  target  to 
predict  performance  against  threats.  VV&A  of  the  test 
resource  requires  target  and  threat  signature  data.  The 
VV&A  process  must  be  accomplished  during  the  test 
program. 

16.  Live  Fire  Testing.  Live  fire  testing  is  a  statutory 
requirement  for  all  Category  I  and  II  acquisition 
programs.  This  requirement  has  grown  directly  out  of 


problems  the  Army  experienced  with  the  Bradley 
Fighting  Vehicle.  The  purpose  of  live  fire  testing  is  to 
determine  if  a  conventional  armament/munition  system 
exhibits  battle-resilient  survivability  (with  primary 
emphasis  on,  but  not  limited  to,  vulnerability)  and  if  an 
armament/munition  or  missile  exhibits  sufficient  lethality 
to  inflict  the  intended  damage. 

Survivability  is  the  capability  of  the  armament/munition 
to  avoid  or  withstand  a  man-made  hostile  environment 
without  suffering  an  abortive  impairment  of  its  ability  to 
accomplish  its  designated  mission.  In  the  case  of 
armament/munitions,  most  are  not  designed  to,  nor 
intended  to,  withstand  battle  damage.  Survivability  for 
guided  armament/munitions  may  be  interpreted  to  mean 
vulnerability  to  countermeasures.  If  vulnerability  testing 
is  required,  the  system  must  be  loaded  or  equipped  with 
all  dangerous  materials  that  would  normally  be  included 
operationally.  All  critical  subsystems  which  could 
contribute  to  the  test  outcome  must  also  be  operating 
under  realistic  conditions. 

Lethality  Testing  is  designed  to  determine  the  ability  of 
an  armament/munition  to  cause  damage  that  will  cause 
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the  loss  or  a  degradation  in  the  ability  of  the  target  system 
to  complete  its  designated  mission.  For  lethality  testing, 
the  armament/munition  must  be  production 
representative.  Targets  must  be  representative  of  the  class 
of  systems  that  includes  the  threat  and  be  sufficiently 
realistic  to  demonstrate  the  lethal  effects  the 
armament/munition  is  designed  to  produce. 

By  Milestone  I,  a  decision  should  be  made  whether  the 
armament/munition  meets  the  legislative  criteria  for  a 
covered  system.  A  mature  strategy  for  live-fire  testing 
should  be  included  in  the  TEMP  by  Milestone  II.  The 
entire  Live  Fire  T&E  program,  to  include  testing, 
evaluation,  and  reporting,  must  be  completed  by 
Milestone  III. 

17.  Joint  Munitions  Test  &  Evaluation  Program 
Office  (CHICKEN  LITTLE).  The  program  office  was 
formed  to  test  and  evaluate  developmental  smart 
armament/munitions  to  reduce  risk  for  the  Army  and  Air 
Force.  Test  methodologies  stress  realism  to  include 
actual  soviet  armor,  countermeasures,  and  environment. 
The  Joint  Service  program  has  been  institutionalized  and 
provides  support  to  smart  armament/munitions 
developers.  The  program  focuses  on  two  areas  of  smart 
armament/munitions;  warhead  effectiveness  and 
seeker/sensor  performance.  Explosively  formed 
projectiles  and  shaped  charge  warheads  are  tested  and 
effectiveness  assessed  against  threat  armored  vehicles 
with  and  without  countermeasures.  The  program  supports 
the  smart  armament/munitions  development  community 
by  developing  and  providing  advanced  methodologies  for 
testing  warheads  and  assessing  target  vulnerabilities. 
Captive  flight  tests  and  signature  measurements  are 
conducted  to  support  the  development  of  submunitions 
that  engage  and  defeat  threat  ground  mobile  vehicles. 
Advanced  methodologies  are  applied  to  evaluate 
developmental  seeker/sensors  to  reduce  risk  and  cost. 
Signatures  of  threat  ground  mobile  systems  are  processed 
and  stored  in  the  Target  Background  Information  Library 
System,  a  comprehensive  interactive  source  of  IR  and 
MMW  target  and  background  data. 

18.  Air  Force  Acquisition  Model  (AFAM).  The  AFAM 
was  developed  as  a  vehicle  where  functional  experts  in 
the  Air  Force  Materiel  Command  (AFMC)  provide 
insight  on  procedures,  best  practices,  and  wisdom  used  to 
complete  acquisition  tasks.  AFAM  is  an  automated 
encyclopedia  that  depicts  the  entire  acquisition  process 
from  cradle-to-grave.  In  addition,  AFAM  contains 
supplement  text  retrieval  system  that  contains  over  65 
DoD  databases  controlled  by  functional  experts.  The 
AFAM  Program  Office  is  located  at  Wright-Patterson 
AFB,  OH,  ASC/CXM,  DSN  785-0416. 

19.  Software  Testing.  In  many  programs  the  highest 
risk  and  most  challenging  aspect  of  the  system  acquisition 
is  the  software  development  effort.  This  is  true  whether 
the  software  is  embedded  in  a  armament/munition  system, 


or  a  standalone  system.  However,  just  as  with  hardware, 
the  key  is  to  use  sound  engineering  and  management 
principles  in  the  design  and  test.  Since  software  is  such  a 
critical  part  of  our  warfighting  capability,  it  is  important 
to  discuss  a  structured  approach  to  it’s  development 
together  with  a  list  of  the  common  software  development 
problems.  This  section  will  discuss  the  issue  of  software 
testing  as  it  relates  to  the  overall  munitions  test  process. 

19.1.  Development  Process.  The  software  development 
process  relates  directly  to  the  basic  scientific  process 
discussed  earlier  in  this  document  and  in  AFI  99-103. 
The  steps  of  the  software  process  are: 

•  Requirements  Analysis 

•  Design  Specification 

•  Testing  and  Integration 

•  Correct  Problems 

•  Feedback 

•  Release  of  Software 

This  process  implements  the  basic  building  block 
approach  from  “components”  to  “subsystems”  to  a 
“system.”  Following  the  initial  development  of  the 
software,  it  is  integrated  with  the  hardware  portion  of  the 
system  and  progresses  through  the  various  test  resource 
categories  and  is  tested  as  an  integrated  system.  An 
important  factor  is  to  not  break  out  the 
armament/munition  system  into  a  program  focusing  on 
separate  hardware  engineering  and  software  engineering. 
Instead,  a  true  systems  engineering  approach  must  be 
adopted.  From  requirements  definition  through  detailed 
design,  integration  and  test,  software  must  be  looked  at  as 
an  integral  part  of  the  system.  Contract  structure  and 
contractor  oversight  must  focus  on  delivery  of  functional 
performance  through  a  balanced  hardware/software 
ensemble. 

19.2.  Requirements  Analysis.  As  with  hardware  the 
user’s  requirements  must  be  defined  and  then  a  system  is 
designed  which  will  meet  those  requirements.  In  the  case 
of  software,  the  requirements  definition  process  can  be 
even  more  difficult  than  that  of  hardware.  Many  times 
the  customer  does  not  have  a  firm  grasp  on  the  software 
requirements  for  the  overall  system.  Analysis  of 
requirements,  in  addition  to  the  user’s  needs  described  in 
functional  and  performance  terms,  must  focus  on  the 
complete  system  capability.  This  includes  both  the 
software  and  the  complete  environment  in  which  the 
system  is  to  operate  The  first  step  of  the  software  test 
process  is  to  fully  define,  as  much  as  possible,  the  firm 
requirements  on  which  to  base  the  design. 

Studies  have  shown  that  roughly  two-thirds  of  the  errors 
in  software  development  are  a  result  of  the  requirements 
analysis  and  design  specification  phases.  There  are 
several  actions  the  developer/tester  can  employ  to  correct 
this  trend. 

One  action  is  to  have  early  and  continuous  user 
involvement  during  the  entire  requirements  process. 
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Requirements  should  be  stated  so  they  are  understandable 
by  the  designer  and  the  developer.  One  measure  of  clarity 
is  whether  or  not  the  requirement  is  testable.  Tools  to  use 
during  the  requirements  definition  process  include  the 
Computer  Aided  Software  Engineering  (CASE) 
programs.  CASE  tools  can  substantially  reduce  many  of 
the  design  and  development  problems  in  large  software 
projects.  Specialized  tools  available  for  the  requirements 
analysis  phase  are  user  interface  prototyping  tools, 
structured  analysis  tools  and  information  modeling  tools. 
Further  information  on  how  the  tools  can  benefit  your 
particular  program  and  provide  answers  to  other  software 
questions  are  available  through  the  following 
organizations. 

SAF/AQKS 
Washington,  DC 
DSN  227-3108 
AF/SCXS 
Washington,  DC 
DSN  223-2699 

AFIT/ENG 

Wright-Patterson  AFB,  OH 
DSN  785-7913 

Air  Force  Software  Technology  Support  Center 
Hill  AFB,  UT 
DSN  458-3207 

Another  tool  to  help  in  the  requirements  definition  effort 
is  the  development  of  early  prototypes.  Prototyping 
allows  an  early  opportunity  for  users  and  developers  to 
come  together  to  identify  promising  technical  aspects  and 
needed  changes  while  still  achievable.  Prototyping  can 
take  many  different  forms.  It  is  best  to  view  a  prototype 
as  a  conceptual  or  technical  test  to  influence  architectures 
and  engineering  development.  The  utility  of  a  prototype 
is  in  verifying  a  concept  or  set  of  procedures,  assessing 
design  approaches,  and  testing  pieces  of  a  developed 
system.  The  prototype,  if  used  properly,  can  reduce  risk, 
shorten  design  to  implementation  cycles,  manage 
requirements,  and  provide  incremental  capabilities  for 
fielding  the  system. 

Rapid  prototyping  is  an  excellent  way  to  ensure  a  clear 
understanding  of  user  requirements.  As  a  minimum,  there 
must  be  agreement  with  the  user  on  the  nature  and 
description  of  the  products  with  which  the  user  will 
directly  interact.  There  must  also  be  understanding  on  the 
tests  and  performance  thresholds  necessary  to  satisfy  user 
requirements. 

19.3.  Design  Specifications.  During  the  design 
specification  phase  the  software  blueprint  is  developed 
showing  what  to  build  and  how  to  build  it.  Modules 
should  be  identified  which  can  be  related  back  to  the 
customer  requirements.  During  this  phase,  the  tester 
should  be  involved  to  determine  the  test  objectives  for 
each  module  and  to  help  determine  the  adequacy  of  the 
design  process  to  answer  the  customer’s  needs.  CASE 


tools  can  also  be  used  effectively  during  this  phase  of 
software  development.  Specific  examples  of  CASE  tools 
in  this  area  are  data  modeling  tools  and  design 
specification  tools. 

Considerations  during  the  design  specification  phase 
include  questions  on  the  ability  to  grow  at  a  later  date. 
This  involves  two  areas.  One  is  the  availability  of  extra 
capacity  and  the  second  is  that  the  design  must  be  flexible 
enough  to  accommodate  future  growth.  Flexibility  can  be 
enhanced  by  the  use  of  modular  design,  software 
partitioning  and  open  architectures. 

19.4.  Testing  and  Integration.  Following  design 
specification  each  module  is  then  coded,  tested  and 
debugged  to  ensure  each  module  performs  as  originally 
designed.  The  tester  must  make  sure  that  all  test 
requirements  identified  in  the  previous  phase  are  fully 
implemented.  The  contractor  must  conduct  Formal 
Qualification  Testing  (FQT)  on  the  various  modules  and 
on  the  integrated  software  product.  These  FQT  tests  must 
stress  the  software  at  the  limits  of  its  specified 
requirements.  Plans  for  FQT  must  be  documented  in  the 
contractor’s  software  development  plan  and  specific  tests 
for  each  requirement  outlined  in  the  contractor’s  software 
test  descriptions.  These  two  documents  are  reviewed  by 
knowledgeable  software  engineers  within  the  program 
office  to  ensure  adequate  testing  will  take  place  on  the 
product. 

The  contractor’s  Software  Quality  Assurance  (SQA) 
should  be  heavily  involved  in  the  FQT  to  ensure  the  test 
plan  and  description  is  followed,  and  that  any 
discrepancies  are  accurately  recorded.  SQA’s  primary 
role  is  to  ensure  the  objectivity  of  the  test. 

The  contractor  must  document  the  traceability  of 
requirements  which  are  satisfied  by  each  test  case.  This 
traceability  is  documented  in  the  contractor’s  software 
test  description. 

19.5.  Correction  of  Software  Faults.  Faults  found 
during  the  testing  of  software  programs  can  fall  into 
several  different  categories.  These  are  requirements 
faults,  features  faults  and  functionality  faults. 
Requirements  faults  are  due  to  the  problem  discussed 
earlier  of  incomplete  or  ambiguous  requirements  which 
are  then  reflected  in  software  specifications. 
Requirements  are  a  major  source  of  faults  ranging  in 
occurrence  from  a  few  percent  to  over  50%  of  faults 
found.  These  faults  typically  lead  to  the  next  category  of 
faults,  the  feature  faults.  Feature  faults  are  comprised  of 
two  categories.  The  easiest  one  to  detect  and  fix  are  the 
missing  features  faults.  Harder  to  identify  and  correct  are 
the  wrong  feature  faults.  Wrong  feature  faults  tend  to 
result  from  improper  design.  One  of  the  easiest  ways  to 
avoid  these  two  categories  of  faults  is  through  the  use  of 
techniques  discussed  earlier  for  thorough  requirements 
definition  and  careful  initial  design. 

The  third  category  of  fault  is  the  functionality  fault. 
Functionality  faults  are  when  the  program’s  actual 
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behavior  is  different  than  the  specified  behavior.  There 
are  many  core  reasons  for  functionality  faults.  Briefly, 
these  are  problems  related  to  structure,  data,  and  coding. 
These  bugs  are  found  by  the  use  of  a  series  of  tests  such 
as  syntax  testing,  domain  testing,  logic  testing,  and  state 
testing. 

One  of  the  main  questions  following  the  successful  fix  of 
any  fault  is  how  much  of  the  code  must  now  be  tested  in 
regression  testing.  Regression  testing  is  that  repetition 
testing  done  to  ensure  that  the  software’s  function  is 
unchanged  except  that  required  to  correct  the  fault. 
Regression  testing  can  be  complicated  if  software 
modularity  is  not  stressed.  If  the  software  is  modular  in 
nature,  the  tester  can  more  easily  test  only  the  affected 
portion  of  the  program. 

The  tester  must  then  ensure  that  the  decision  makers  are 
aware  of  the  categories  of  software  faults  discovered 
ranging  from  minor  to  critical  and  that  they  can  be 
confident  that  corrections  have  been  implemented 
successfully. 

19.6.  Feedback/Metrics.  It  is  important  that  the 
program  office  track  contractor  progress  during  the  design 
and  test  of  any  software  product.  Meaningful  information 
on  progress  must  be  forwarded  to  the  appropriate  decision 
maker  so  that  informed  decisions  can  occur  regarding  the 


development  effort.  Metrics  must  be  utilized  which 
actually  give  the  leadership  a  true  picture  of  the  progress. 
In  the  past,  metrics  such  as  “lines  of  code  tested”  have 
been  used  as  an  indicator.  The  test  organization  must  be 
careful  in  providing  metrics  such  as  this,  because  in  many 
cases  this  does  not  accurately  depict  the  true  status. 
Instead,  requirements  traceability  and  software  error 
correction  metrics  may  serve  to  point  out  potential  or  real 
problems  the  contractor  is  experiencing.  For  guidance  on 
software  metrics  policy  and  implementation,  refer  to 
SAF/AQ  Acquisition  Policy  93M-017,  dated  16  Feb  94. 
19.7.  Release  of  Software.  As  stated  previously,  after 
the  “standalone”  software  testing,  the  software  is 
integrated  with  the  hardware  and  the  system  then  goes 
through  the  applicable  test  resource  for  a  complete  test  of 
integrated  components,  subsystems  and  the  system. 
During  this  phase  of  testing  it  is  very  important  to 
remember  that  software  is  just  as  critical  to  the 
performance  of  the  overall  system  as  hardware.  Problems 
in  function  and  performance  involving  software  should 
not  be  looked  at  as  “just  a  software  fix”  but  should  be 
looked  at  from  the  standpoint  of  the  integrated  system. 
Software  development  should  not  be  driven  by  the 
hardware  development  effort  but  instead  should  be  looked 
at  as  an  equal  and  very  critical  part  of  the  process. 


HOWARD  W.  LEAF,  Lt  General,  USAF  (Retired) 
Director,  Test  and  Evaluation 
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References 

AFI  99-101,  Development  Test  and  Evaluation.  Provides  guidance  and  procedures  for  the  DT&E  of  Air  Force  systems. 

AFI  99-102,  Operational  Test  and  Evaluation.  Provides  guidance  and  procedures  for  the  OT&E  of  Air  Force  systems. 

AFI  99-103,  Air  Force  Test  and  Evaluation  Process.  Directs  use  of  and  describes  the  Air  Force  Test  Process. 

AFPD  99-1,  Test  and  Evaluation.  Implements  the  Air  Force  Test  Process. 

Air  Force  Acquisition  Policy  Letter  93M-017,  Software  Metrics  Policy.  Establishes  Software  Metrics  policy. 

AFI  63-104,  Aircraft-Stores  Certification  Program.  Defines  the  aircraft-stores  certification  (SEEK  EAGLE)  process. 

SEEK  EAGLE  Engineering/Test  Capabilities  Handbook.  Identifies  the  location,  primary  mission,  and  major  aircraft-store 
certification  resources. 

AFI  99-105,  Live  Fire.  Defines  the  Air  Force  Live  Fire  test  program. 


Abbreviations  and  Acronyms 

AFAM 

Air  Force  Acquisition  Model 

AFB 

Air  Force  Base 

AFI 

Air  Force  Instruction 

AFMC 

Air  Force  Materiel  Command 

AFOTEC 

Air  Force  Operational  Test  and  Evaluation  Center 

AFSEO 

Air  Force  SEEK  EAGLE  Office 

ASTE 

Armament  Systems  Test  Environment 

CASE 

Computer  Aided  Software  Engineering 

CE&D 

Concept  Exploration  and  Definition 

CFD 

Computational  Fluid  Dynamics 

COEA 

Cost  &  Operational  Effectiveness  Analysis 

COI 

Critical  Operational  Issue 

CONOPS 

Concept  of  Operations 

CSAF 

Chief  of  Staff,  Air  Force 

Dem/Val 

Demon  strati  on  /  V  alidati  on 

DIA 

Defense  Intelligence  Agency 

DoD 

Department  of  Defense 

DOF 

degree-of-freedom 

DSM 

Digital  System  Model 

DT&E 

Developmental  Test  and  Evaluation 

EM 

Electromagnetic 

E&MD 

Engineering  and  Manufacturing  Development 

EMTE 

ElectromagneticTest  Environment 

EO 

Electro-Optic 

FC 

Flight  Clearance 

FCSC 

Freeman  Computer  Science  Center 

FQT 

Formal  Qualification  Testing 

GPS 

Global  Positioning  System 

GTR 

Gulf  Test  Range 

GWEF 

Guided  Weapons  Evaluation  Facility 

HITL 

Hardware-In-The-Loop 

IOT&E 

Initial  Operational  Test  and  Evaluation 
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IR 

1ST 

ISTF 

JMEM 

JTCG/ME 

MMW 

MNS 

MOA 

MOE 

MOP 

NM 

OAR 

OFP 

OPR 

ORD 

OT&E 

PGM 

PRIMES 

RAMS 

RATSCAT 

RCS 

RF 

RTO 

SE 

SEMSS 

SER 

SFTC 

SIL 

SMM 

SPP 

SQA 

T&E 

TEMP 

TO 

TPA 

TPP 

TPWG 

TSPI 

UTTR 

uv 

V&A 

VV&A 

WIP 

WTT 


Infrared 

Installed  Systems  Testing 
Installed  Systems  Testing  Facility 
Joint  Munitions  Effectiveness  Manual 

Joint  Technical  Coordinating  Group  for  Munitions  Effectiveness 

Millimeter  Wave 

Mission  Need  Statement 

Military  Operating  Area 

Measure  of  Effectiveness 

Measure  of  Performance 

Nautical  Miles 

Open  Air  Range 

Operational  Flight  Program 

Office  of  Primary  Responsibility 

Operational  Requirements  Document 

Operational  Test  and  Evaluation 

Precision  Guided  Munition 

Preflight  Integration  of  Munitions  and  Electronic  Systems 

RATSCAT  Advanced  Measurement  System 

Radar  Target  Scatter  Facility 

Radar  Cross  Section 

Radio  Frequency 

Responsible  Test  Organization 

SEEK  EAGLE 

SEEK  EAGLE  Management  Support  System 

SEEK  EAGLE  Request 

Single-Face-to-Customer 

System  Integration  Laboratory 

System  Maturity  Matrix 

System  Performance  Parameter 

Software  Quality  Assurance 

Test  and  Evaluation 

Test  and  Evaluation  Master  Plan 

Technical  Order 

Test  Process  Archive 

Technical  Performance  Parameter 

Test  Planning  Working  Group 

Time,  Space,  Position  Information 

Utah  Test  and  Training  Range 

Ultraviolet 

Validation  and  Accreditation 
Verification,  Validation,  and  Accreditation 
Weapon  Integration  Plan 
Wind  Tunnel  Testing 


